[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:20 dist-test-slave-q9h9 krb5kdc[16289](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:20 dist-test-slave-q9h9 krb5kdc[16289](info): set up 2 sockets
Aug 12 14:42:20 dist-test-slave-q9h9 krb5kdc[16289](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 14:42:21.227274 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:21 dist-test-slave-q9h9 krb5kdc[16289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:21.338524 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33985
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:21.420033 16309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:21.420212 16309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:21.420233 16309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:21.421304 16309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:21.421334 16309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:21.421347 16309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:21.421358 16309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:21.421368 16309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:21.422982 16309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:21.423233 16309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:21.423408 16309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:21.428195 16318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:21.428220 16325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:21.428284 16309 server_base.cc:1034] running on GCE node
W20260812 14:42:21.428220 16317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:21.428611 16309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:21.428949 16309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:21.430063 16309 hybrid_clock.cc:648] HybridClock initialized: now 1786545741430023 us; error 53 us; skew 500 ppm
Aug 12 14:42:21 dist-test-slave-q9h9 krb5kdc[16289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545741, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:21.435066 16309 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:21.435500 16309 webserver.cc:460] Webserver started at http://127.15.227.254:44633/ using document root <none> and password file <none>
I20260812 14:42:21.435681 16309 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:21.435719 16309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:21.435802 16309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:21.436749 16309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "5186c51264324d4398bcbc2eac8acec8"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "2e2810d92fb813035ab6ad8a6d3d4274"
server_key_iv: "07e3d263ee37ad85a5f4a6af3444663a"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.437034 16309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "5186c51264324d4398bcbc2eac8acec8"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "2e2810d92fb813035ab6ad8a6d3d4274"
server_key_iv: "07e3d263ee37ad85a5f4a6af3444663a"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.438319 16309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:21.438961 16334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:21.439152 16309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:21.439224 16309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "5186c51264324d4398bcbc2eac8acec8"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "2e2810d92fb813035ab6ad8a6d3d4274"
server_key_iv: "07e3d263ee37ad85a5f4a6af3444663a"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.439276 16309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:21.452639 16309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:21.454414 16309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:21.454512 16309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:21.458557 16309 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:33985
I20260812 14:42:21.458603 16434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:33985 every 8 connection(s)
I20260812 14:42:21.458851 16309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:21.459800 16435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:21.462218 16435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8: Bootstrap starting.
I20260812 14:42:21.462597 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 16309
I20260812 14:42:21.462699 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:21.462921 16435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:21.463254 16435 log.cc:824] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:21.463486 16271 external_mini_cluster.cc:1428] Setting key 04023af305923929709c87a04717685e
I20260812 14:42:21.464088 16435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8: No bootstrap required, opened a new log
I20260812 14:42:21.466439 16435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186c51264324d4398bcbc2eac8acec8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33985 } }
I20260812 14:42:21.466604 16435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:21.466648 16435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5186c51264324d4398bcbc2eac8acec8, State: Initialized, Role: FOLLOWER
I20260812 14:42:21.466765 16435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186c51264324d4398bcbc2eac8acec8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33985 } }
I20260812 14:42:21.466899 16435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:21.466948 16435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:21.466996 16435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:21.467958 16435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186c51264324d4398bcbc2eac8acec8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33985 } }
I20260812 14:42:21.468078 16435 leader_election.cc:302] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5186c51264324d4398bcbc2eac8acec8; no voters:
I20260812 14:42:21.468236 16435 leader_election.cc:288] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:21.468278 16447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:21.468430 16447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [term 1 LEADER]: Becoming Leader. State: Replica: 5186c51264324d4398bcbc2eac8acec8, State: Running, Role: LEADER
I20260812 14:42:21.468528 16435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:21.468575 16447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186c51264324d4398bcbc2eac8acec8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33985 } }
I20260812 14:42:21.468935 16447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5186c51264324d4398bcbc2eac8acec8. Latest consensus state: current_term: 1 leader_uuid: "5186c51264324d4398bcbc2eac8acec8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186c51264324d4398bcbc2eac8acec8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33985 } } }
I20260812 14:42:21.469147 16447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:21.469080 16453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5186c51264324d4398bcbc2eac8acec8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5186c51264324d4398bcbc2eac8acec8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33985 } } }
I20260812 14:42:21.469547 16453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:21.469832 16467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:21.470619 16467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:21.472062 16467 catalog_manager.cc:1347] Generated new cluster ID: b7d1ae73c7884e63ad53bebf1d4ace16
I20260812 14:42:21.472112 16467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:42:21 dist-test-slave-q9h9 krb5kdc[16289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:21.480722 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:21.464291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58576 (local address 127.15.227.254:33985)
0812 14:42:21.464497 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:21.464506 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:21.464719 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:42:21.466931 (+ 2212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:21.466938 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:21.466990 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:21.467114 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:21.469223 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.470674 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.472067 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.472654 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.472751 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:21.476845 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:21.476849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:21.476850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:21.476861 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:21.478833 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:21.479285 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:21.479289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:21.479289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:21.479405 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:21.479645 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:21.479646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:21.479646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:21.479937 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:21.480085 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:21.480326 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 14:42:21.480459 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260812 14:42:21.490005 16467 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:21.490540 16467 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:21.497058 16467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5186c51264324d4398bcbc2eac8acec8: Generated new TSK 0
I20260812 14:42:21.497215 16467 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:21.522250 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33985
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:21.658609 16491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:21.658806 16491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:21.658833 16491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:21.660053 16491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:21.660099 16491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:21.660132 16491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:21.661777 16491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33985
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:21.662047 16491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:21.662276 16491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:21.667547 16506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:21.667593 16513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:21.667740 16505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:21.667745 16491 server_base.cc:1034] running on GCE node
I20260812 14:42:21.667976 16491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:21.668200 16491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:21.669334 16491 hybrid_clock.cc:648] HybridClock initialized: now 1786545741669317 us; error 34 us; skew 500 ppm
Aug 12 14:42:21 dist-test-slave-q9h9 krb5kdc[16289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545741, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:21.676020 16491 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:21.676451 16491 webserver.cc:460] Webserver started at http://127.15.227.193:43203/ using document root <none> and password file <none>
I20260812 14:42:21.676657 16491 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:21.676710 16491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:21.676839 16491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:21.677839 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "f25378cbec1441e59f53fdd512c55592"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "7377a4bd9d492cda927dcd063456542e"
server_key_iv: "33e971c07fa5b7b4047ce854ce8d281b"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.678210 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "f25378cbec1441e59f53fdd512c55592"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "7377a4bd9d492cda927dcd063456542e"
server_key_iv: "33e971c07fa5b7b4047ce854ce8d281b"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.679510 16491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:21.680171 16530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:21.680331 16491 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:21.680418 16491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "f25378cbec1441e59f53fdd512c55592"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "7377a4bd9d492cda927dcd063456542e"
server_key_iv: "33e971c07fa5b7b4047ce854ce8d281b"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.680485 16491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:21.694839 16491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:21.696616 16491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:21.696750 16491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:21.696961 16491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:21.697247 16491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:21.697300 16491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:21.697332 16491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:21.697364 16491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:21.709677 16491 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:34667
I20260812 14:42:21.709710 16743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:34667 every 8 connection(s)
I20260812 14:42:21.710068 16491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:42:21 dist-test-slave-q9h9 krb5kdc[16289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545741, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:21.718917 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 16491
I20260812 14:42:21.719018 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:21.719187 16271 external_mini_cluster.cc:1428] Setting key 595d8e97b76306f0b857e72c1e7c7e04
I20260812 14:42:21.720356 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:21.710933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59915 (local address 127.15.227.254:33985)
0812 14:42:21.710979 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:21.710981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:21.711243 (+ 262us) server_negotiation.cc:407] Connection header received
0812 14:42:21.711906 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:21.711907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:21.711917 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:21.711972 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:21.713099 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.713603 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.714819 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.715025 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.715080 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:21.717415 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:21.717417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:21.717417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:21.717424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:21.718796 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:21.719235 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:21.719236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:21.719237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:21.719304 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:21.719622 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:21.719623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:21.719623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:21.719743 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:21.719809 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:21.720197 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 14:42:21.720293 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:42:21.720592 16748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:21.710908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33985 (local address 127.15.227.193:59915)
0812 14:42:21.711131 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:21.711168 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:21.711822 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:21.712028 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:21.712031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:21.712757 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:21.713023 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:21.713028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.713672 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.713673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:21.714711 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:21.714714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.715093 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.715094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:21.715255 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:21.715773 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:21.715781 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:21.717312 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:21.718885 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:21.718892 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:21.718895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:21.719168 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:21.719438 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:21.719439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:21.719439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:21.719560 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:21.719886 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:21.719894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:21.720048 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 14:42:21.720172 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 14:42:21.720348 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":76,"threads_started":1}
I20260812 14:42:21.721145 16744 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33985
I20260812 14:42:21.721239 16744 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:21.721419 16744 heartbeater.cc:511] Master 127.15.227.254:33985 requested a full tablet report, sending...
I20260812 14:42:21.722406 16368 ts_manager.cc:194] Registered new tserver with Master: f25378cbec1441e59f53fdd512c55592 (127.15.227.193:34667)
I20260812 14:42:21.723469 16368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:59915
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:21.745806 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33985
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:21.863292 16755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:21.863480 16755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:21.863504 16755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:21.864774 16755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:21.864820 16755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:21.864845 16755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:21.866525 16755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33985
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:21.866765 16755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:21.866983 16755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:21.872889 16763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:21.872859 16766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:21.873013 16770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:21.872997 16755 server_base.cc:1034] running on GCE node
I20260812 14:42:21.873219 16755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:21.873464 16755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:21.874593 16755 hybrid_clock.cc:648] HybridClock initialized: now 1786545741874579 us; error 30 us; skew 500 ppm
Aug 12 14:42:21 dist-test-slave-q9h9 krb5kdc[16289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545741, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:21.877575 16755 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:21.877964 16755 webserver.cc:460] Webserver started at http://127.15.227.194:36723/ using document root <none> and password file <none>
I20260812 14:42:21.878171 16755 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:21.878213 16755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:21.878312 16755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:21.879370 16755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "67d99e1a2c3e4d65ae6e7f82ffe92a8c"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "f7cb70e49adb93c9e40e13fd5dd4babe"
server_key_iv: "abdef8bfa4cac00ba05f9ff70193f6da"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.879702 16755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "67d99e1a2c3e4d65ae6e7f82ffe92a8c"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "f7cb70e49adb93c9e40e13fd5dd4babe"
server_key_iv: "abdef8bfa4cac00ba05f9ff70193f6da"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.880990 16755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:21.881803 16786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:21.881956 16755 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:21.882061 16755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "67d99e1a2c3e4d65ae6e7f82ffe92a8c"
format_stamp: "Formatted at 2026-08-12 14:42:21 on dist-test-slave-q9h9"
server_key: "f7cb70e49adb93c9e40e13fd5dd4babe"
server_key_iv: "abdef8bfa4cac00ba05f9ff70193f6da"
server_key_version: "encryptionkey@0"
I20260812 14:42:21.882120 16755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:21.894157 16755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:21.896651 16755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:21.896857 16755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:21.897248 16755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:21.897615 16755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:21.897658 16755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:21.897689 16755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:21.897710 16755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:21.914631 16755 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:39343
I20260812 14:42:21.914724 16982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:39343 every 8 connection(s)
I20260812 14:42:21.915066 16755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:21.920368 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 16755
I20260812 14:42:21.920455 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:21.920593 16271 external_mini_cluster.cc:1428] Setting key dde15aceb0f1b9e3ce2439d777fe9094
Aug 12 14:42:21 dist-test-slave-q9h9 krb5kdc[16289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545741, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:21.925525 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:21.915895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:55441 (local address 127.15.227.254:33985)
0812 14:42:21.915944 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:21.915946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:21.916260 (+ 314us) server_negotiation.cc:407] Connection header received
0812 14:42:21.917059 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:21.917061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:21.917076 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:21.917131 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:21.918404 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.918914 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.920003 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.920202 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.920259 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:21.922835 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:21.922838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:21.922838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:21.922846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:21.924226 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:21.924638 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:21.924639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:21.924639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:21.924690 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:21.924936 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:21.924937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:21.924937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:21.925059 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:21.925106 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:21.925401 (+ 295us) server_negotiation.cc:299] Negotiation successful
0812 14:42:21.925464 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:42:21.925766 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:21.915852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33985 (local address 127.15.227.194:55441)
0812 14:42:21.916103 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:21.916142 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:21.916942 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:21.917217 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:21.917222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:21.917850 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:21.918288 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:21.918295 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.919000 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.919001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:21.919913 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:21.919916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:21.920273 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:21.920274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:21.920432 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:21.920974 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:21.920982 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:21.922744 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:21.924301 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:21.924307 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:21.924309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:21.924574 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:21.924745 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:21.924746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:21.924746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:21.924873 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:21.925113 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:21.925119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:21.925278 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 14:42:21.925377 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 14:42:21.925542 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":74,"threads_started":1}
I20260812 14:42:21.926231 16984 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33985
I20260812 14:42:21.926324 16984 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:21.926486 16984 heartbeater.cc:511] Master 127.15.227.254:33985 requested a full tablet report, sending...
I20260812 14:42:21.926867 16368 ts_manager.cc:194] Registered new tserver with Master: 67d99e1a2c3e4d65ae6e7f82ffe92a8c (127.15.227.194:39343)
I20260812 14:42:21.927215 16368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:55441
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:21.945026 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33985
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:22.066843 16995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:22.067037 16995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:22.067080 16995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:22.068243 16995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:22.068326 16995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:22.068431 16995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:22.070281 16995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33985
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:22.070505 16995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:22.070710 16995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:22.076725 17003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:22.076773 17004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:22.076822 17008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:22.076905 16995 server_base.cc:1034] running on GCE node
I20260812 14:42:22.077091 16995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:22.077328 16995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:22.078454 16995 hybrid_clock.cc:648] HybridClock initialized: now 1786545742078445 us; error 27 us; skew 500 ppm
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[16289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545742, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:22.081496 16995 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:22.081873 16995 webserver.cc:460] Webserver started at http://127.15.227.195:34959/ using document root <none> and password file <none>
I20260812 14:42:22.082068 16995 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:22.082113 16995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:22.082207 16995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:22.083320 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "6241fdaf5c424f9f8c9e49f732e9e9a6"
format_stamp: "Formatted at 2026-08-12 14:42:22 on dist-test-slave-q9h9"
server_key: "75d656c05992cb118337b8daaf9c75fa"
server_key_iv: "cc3a414dd643d30d6bcc9449207e6b48"
server_key_version: "encryptionkey@0"
I20260812 14:42:22.083673 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "6241fdaf5c424f9f8c9e49f732e9e9a6"
format_stamp: "Formatted at 2026-08-12 14:42:22 on dist-test-slave-q9h9"
server_key: "75d656c05992cb118337b8daaf9c75fa"
server_key_iv: "cc3a414dd643d30d6bcc9449207e6b48"
server_key_version: "encryptionkey@0"
I20260812 14:42:22.084956 16995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:22.085716 17024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:22.085855 16995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:22.085937 16995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "6241fdaf5c424f9f8c9e49f732e9e9a6"
format_stamp: "Formatted at 2026-08-12 14:42:22 on dist-test-slave-q9h9"
server_key: "75d656c05992cb118337b8daaf9c75fa"
server_key_iv: "cc3a414dd643d30d6bcc9449207e6b48"
server_key_version: "encryptionkey@0"
I20260812 14:42:22.085994 16995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:22.103807 16995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:22.105880 16995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:22.106107 16995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:22.106498 16995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:22.106874 16995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:22.106911 16995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:22.106938 16995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:22.106953 16995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:22.122242 16995 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:41255
I20260812 14:42:22.122445 17255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:41255 every 8 connection(s)
I20260812 14:42:22.122639 16995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[16289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545742, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:22.130330 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 16995
I20260812 14:42:22.130430 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:22.130609 16271 external_mini_cluster.cc:1428] Setting key 5ffc7cea73b8e13ba91d92f085b65fd0
I20260812 14:42:22.132840 16442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:22.123481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:36391 (local address 127.15.227.254:33985)
0812 14:42:22.123528 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:22.123529 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:22.123858 (+ 329us) server_negotiation.cc:407] Connection header received
0812 14:42:22.124592 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:22.124593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:22.124604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:22.124655 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:22.125684 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.126190 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.127210 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.127385 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.127429 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:22.129933 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:22.129935 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:22.129935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:22.129943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:22.131704 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:22.132132 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:22.132133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:22.132133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:22.132190 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:22.132378 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:22.132378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:22.132378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:22.132470 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:22.132512 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:22.132722 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 14:42:22.132778 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:42:22.133067 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:22.123430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33985 (local address 127.15.227.195:36391)
0812 14:42:22.123721 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:22.123755 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:22.124502 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:22.124709 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:22.124712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:22.125323 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:22.125612 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:22.125617 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.126257 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.126258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:22.127150 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:22.127152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.127542 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.127544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:22.127716 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:22.128193 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:22.128202 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:22.129827 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:22.131794 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:22.131803 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:22.131804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:22.132069 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:22.132242 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:22.132242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:22.132242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:22.132336 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:22.132510 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:22.132513 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:22.132620 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 14:42:22.132704 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:42:22.132818 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":171,"threads_started":1}
I20260812 14:42:22.133540 17259 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33985
I20260812 14:42:22.133632 17259 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:22.133805 17259 heartbeater.cc:511] Master 127.15.227.254:33985 requested a full tablet report, sending...
I20260812 14:42:22.134159 16368 ts_manager.cc:194] Registered new tserver with Master: 6241fdaf5c424f9f8c9e49f732e9e9a6 (127.15.227.195:41255)
I20260812 14:42:22.134514 16368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:36391
I20260812 14:42:22.142616 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[16289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[16289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:22.168550 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:22.160445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.15.227.193:34667)
0812 14:42:22.160605 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:22.160610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:22.160622 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:42:22.160677 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:22.160679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:22.160694 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:22.160808 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:22.161304 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.162371 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.163079 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.163453 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.163505 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:22.165585 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:22.165593 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:22.165593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:22.165600 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:22.167357 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:22.167699 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:22.167701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:22.167701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:22.167837 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:22.167983 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:22.167984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:22.167984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:22.168110 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:22.168239 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:22.168334 (+ 95us) server_negotiation.cc:299] Negotiation successful
0812 14:42:22.168395 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260812 14:42:22.168928 16569 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34696
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[16289](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[16289](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:22.193874 17272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:22.187077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.15.227.193:34667)
0812 14:42:22.187142 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:22.187144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:22.187188 (+ 44us) server_negotiation.cc:407] Connection header received
0812 14:42:22.187247 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:22.187248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:22.187257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:22.187301 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:22.187782 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.188345 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.189133 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:22.189392 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:22.189446 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:22.191702 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:22.191703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:22.191703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:22.191711 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:22.192923 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:22.193194 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:22.193195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:22.193196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:22.193240 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:22.193369 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:22.193369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:22.193370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:22.193442 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:22.193681 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:22.193724 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 14:42:22.193809 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:42:22.194345 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 16491
I20260812 14:42:22.212359 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 16755
I20260812 14:42:22.233541 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 16995
I20260812 14:42:22.259250 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 16309
2026-08-12T14:42:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2257 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[17287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[17287](info): set up 2 sockets
Aug 12 14:42:22 dist-test-slave-q9h9 krb5kdc[17287](info): commencing operation
krb5kdc: starting...
W20260812 14:42:23.465160 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:23 dist-test-slave-q9h9 krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:23.573577 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:44061
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:44061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:23.653422 17310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:23.653604 17310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:23.653631 17310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:23.654804 17310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:23.654842 17310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:23.654861 17310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:23.654877 17310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:23.654893 17310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:23.656323 17310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:44061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:44061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:23.656558 17310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:23.656755 17310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:23.661092 17320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:23.661115 17325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:23.661298 17310 server_base.cc:1034] running on GCE node
W20260812 14:42:23.661211 17321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:23.661563 17310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:23.661835 17310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:23.662971 17310 hybrid_clock.cc:648] HybridClock initialized: now 1786545743662957 us; error 26 us; skew 500 ppm
Aug 12 14:42:23 dist-test-slave-q9h9 krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:23.665259 17310 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:23.665639 17310 webserver.cc:460] Webserver started at http://127.15.227.254:38773/ using document root <none> and password file <none>
I20260812 14:42:23.665829 17310 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:23.665867 17310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:23.665956 17310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:23.666891 17310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "0cfd4cb04d33468682543ec91ff46ff8"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "6315ffe990f11890e3d87c9d7f2c18fd"
server_key_iv: "978d476804b9221ffbbac3cc3f0d94a4"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.667166 17310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "0cfd4cb04d33468682543ec91ff46ff8"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "6315ffe990f11890e3d87c9d7f2c18fd"
server_key_iv: "978d476804b9221ffbbac3cc3f0d94a4"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.668272 17310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:42:23.668891 17339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:23.669050 17310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:23.669111 17310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "0cfd4cb04d33468682543ec91ff46ff8"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "6315ffe990f11890e3d87c9d7f2c18fd"
server_key_iv: "978d476804b9221ffbbac3cc3f0d94a4"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.669166 17310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:23.680709 17310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:23.682243 17310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:23.682344 17310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:23.686167 17310 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:44061
I20260812 14:42:23.686244 17435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:44061 every 8 connection(s)
I20260812 14:42:23.686498 17310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:23.687247 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 17310
I20260812 14:42:23.687342 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:23.687379 17436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:23.687497 16271 external_mini_cluster.cc:1428] Setting key 493fd5c3badb32bac9f256b7550632d7
I20260812 14:42:23.689997 17436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8: Bootstrap starting.
I20260812 14:42:23.690624 17436 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:23.690884 17436 log.cc:824] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:23.691488 17436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8: No bootstrap required, opened a new log
Aug 12 14:42:23 dist-test-slave-q9h9 krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:23.693727 17436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfd4cb04d33468682543ec91ff46ff8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 44061 } }
I20260812 14:42:23.693848 17436 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:23.693868 17436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cfd4cb04d33468682543ec91ff46ff8, State: Initialized, Role: FOLLOWER
I20260812 14:42:23.693969 17436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfd4cb04d33468682543ec91ff46ff8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 44061 } }
I20260812 14:42:23.694069 17436 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:23.694111 17436 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:23.694160 17436 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:23.694752 17436 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfd4cb04d33468682543ec91ff46ff8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 44061 } }
I20260812 14:42:23.694854 17436 leader_election.cc:302] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cfd4cb04d33468682543ec91ff46ff8; no voters:
I20260812 14:42:23.695015 17436 leader_election.cc:288] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:23.695124 17448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:23.695269 17436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:23.695295 17448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [term 1 LEADER]: Becoming Leader. State: Replica: 0cfd4cb04d33468682543ec91ff46ff8, State: Running, Role: LEADER
I20260812 14:42:23.695413 17448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfd4cb04d33468682543ec91ff46ff8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 44061 } }
I20260812 14:42:23.695820 17452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cfd4cb04d33468682543ec91ff46ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfd4cb04d33468682543ec91ff46ff8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 44061 } } }
I20260812 14:42:23.695895 17452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:23.696053 17455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cfd4cb04d33468682543ec91ff46ff8. Latest consensus state: current_term: 1 leader_uuid: "0cfd4cb04d33468682543ec91ff46ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cfd4cb04d33468682543ec91ff46ff8" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 44061 } } }
I20260812 14:42:23.696108 17455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:23.696305 17463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:23.696796 17463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:23.697767 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:23.687976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53818 (local address 127.15.227.254:44061)
0812 14:42:23.688164 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:23.688171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:23.688193 (+ 22us) server_negotiation.cc:407] Connection header received
0812 14:42:23.688645 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:23.688652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:23.688694 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:23.688809 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:23.689403 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:23.690382 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:23.691196 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:23.691602 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:23.691736 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:23.694162 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:23.694166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:23.694166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:23.694177 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:23.696071 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:23.696458 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:23.696461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:23.696461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:23.696560 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:23.696764 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:23.696765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:23.696765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:23.697057 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:23.697213 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:23.697460 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 14:42:23.697568 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260812 14:42:23.699074 17463 catalog_manager.cc:1347] Generated new cluster ID: 28ac7ad25b1548f3ab45addaa127d374
I20260812 14:42:23.699136 17463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:23.712718 17463 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:23.713366 17463 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:23.720870 17463 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cfd4cb04d33468682543ec91ff46ff8: Generated new TSK 0
I20260812 14:42:23.721020 17463 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:23.750851 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:44061
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:23.830436 17484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:23.830626 17484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:23.830650 17484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:23.831777 17484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:23.831820 17484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:23.831844 17484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:23.833318 17484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:44061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:23.833532 17484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:23.833748 17484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:23.838479 17497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:23.838457 17501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:23.838594 17484 server_base.cc:1034] running on GCE node
W20260812 14:42:23.838681 17498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:23.838882 17484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:23.839094 17484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:23.840235 17484 hybrid_clock.cc:648] HybridClock initialized: now 1786545743840222 us; error 29 us; skew 500 ppm
Aug 12 14:42:23 dist-test-slave-q9h9 krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:23.842623 17484 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:23.843009 17484 webserver.cc:460] Webserver started at http://127.15.227.193:42093/ using document root <none> and password file <none>
I20260812 14:42:23.843235 17484 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:23.843283 17484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:23.843408 17484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:23.844345 17484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "322cba8524d54f6e9306a966b834bba8"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "cfda1898a19bcb5de241ae3d073b35c8"
server_key_iv: "6f13d30fff72b2dc782bf32a09750016"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.844648 17484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "322cba8524d54f6e9306a966b834bba8"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "cfda1898a19bcb5de241ae3d073b35c8"
server_key_iv: "6f13d30fff72b2dc782bf32a09750016"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.845888 17484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:42:23.846673 17516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:23.846820 17484 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:23.846901 17484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "322cba8524d54f6e9306a966b834bba8"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "cfda1898a19bcb5de241ae3d073b35c8"
server_key_iv: "6f13d30fff72b2dc782bf32a09750016"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.846961 17484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:23.863417 17484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:23.865156 17484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:23.865283 17484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:23.865466 17484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:23.865734 17484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:23.865767 17484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:23.865794 17484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:23.865810 17484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:23.871181 17484 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:36699
I20260812 14:42:23.871259 17749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:36699 every 8 connection(s)
I20260812 14:42:23.871577 17484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:23.875123 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 17484
I20260812 14:42:23.875205 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:23.875352 16271 external_mini_cluster.cc:1428] Setting key e5f032b28bb1e177c86b84172d111fe2
Aug 12 14:42:23 dist-test-slave-q9h9 krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:23.880775 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:23.872209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46657 (local address 127.15.227.254:44061)
0812 14:42:23.872253 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:23.872256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:23.872430 (+ 174us) server_negotiation.cc:407] Connection header received
0812 14:42:23.873500 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:23.873502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:23.873515 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:23.873567 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:23.874179 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:23.874689 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:23.875872 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:23.876184 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:23.876263 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:23.878310 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:23.878312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:23.878312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:23.878320 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:23.879534 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:23.879907 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:23.879908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:23.879908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:23.879964 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:23.880177 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:23.880178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:23.880179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:23.880287 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:23.880342 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:23.880594 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 14:42:23.880685 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:42:23.880918 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:23.872186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:44061 (local address 127.15.227.193:46657)
0812 14:42:23.872318 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:23.872354 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:23.873386 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:23.873631 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:23.873636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:23.873879 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:23.874079 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:23.874083 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:23.874770 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:23.874771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:23.875803 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:23.875806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:23.876269 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:23.876270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:23.876374 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:23.876795 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:23.876803 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:23.878228 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:23.879610 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:23.879617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:23.879619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:23.879841 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:23.880019 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:23.880019 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:23.880020 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:23.880119 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:23.880342 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:23.880349 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:23.880478 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 14:42:23.880562 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:42:23.880719 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260812 14:42:23.881441 17751 heartbeater.cc:347] Connected to a master server at 127.15.227.254:44061
I20260812 14:42:23.881546 17751 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:23.881737 17751 heartbeater.cc:511] Master 127.15.227.254:44061 requested a full tablet report, sending...
I20260812 14:42:23.882282 17362 ts_manager.cc:194] Registered new tserver with Master: 322cba8524d54f6e9306a966b834bba8 (127.15.227.193:36699)
I20260812 14:42:23.883213 17362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:46657
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:23.898921 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:44061
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:23.978132 17759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:23.978308 17759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:23.978330 17759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:23.979445 17759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:23.979480 17759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:23.979504 17759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:23.980940 17759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:44061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:23.981117 17759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:23.981292 17759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:23.985980 17768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:23.985982 17781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:23.986122 17759 server_base.cc:1034] running on GCE node
W20260812 14:42:23.985982 17770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:23.986423 17759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:23.986653 17759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:23.987790 17759 hybrid_clock.cc:648] HybridClock initialized: now 1786545743987777 us; error 29 us; skew 500 ppm
Aug 12 14:42:23 dist-test-slave-q9h9 krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:23.990094 17759 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:23.990518 17759 webserver.cc:460] Webserver started at http://127.15.227.194:45507/ using document root <none> and password file <none>
I20260812 14:42:23.990728 17759 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:23.990770 17759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:23.990893 17759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:23.991839 17759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "ebc81ad902bd44698ce8aed0218476e7"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "bd50a4bb218ffa7aa1e13ce4d14767cf"
server_key_iv: "3b1a650cbfcfb7e876a6ffe8b1de71ec"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.992168 17759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "ebc81ad902bd44698ce8aed0218476e7"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "bd50a4bb218ffa7aa1e13ce4d14767cf"
server_key_iv: "3b1a650cbfcfb7e876a6ffe8b1de71ec"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.993330 17759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:23.993942 17794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:23.994105 17759 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:23.994202 17759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "ebc81ad902bd44698ce8aed0218476e7"
format_stamp: "Formatted at 2026-08-12 14:42:23 on dist-test-slave-q9h9"
server_key: "bd50a4bb218ffa7aa1e13ce4d14767cf"
server_key_iv: "3b1a650cbfcfb7e876a6ffe8b1de71ec"
server_key_version: "encryptionkey@0"
I20260812 14:42:23.994269 17759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:24.013700 17759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:24.016181 17759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:24.016355 17759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:24.016613 17759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:24.016948 17759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:24.016984 17759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:24.017015 17759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:24.017040 17759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:24.022967 17759 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:42115
I20260812 14:42:24.023015 18012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:42115 every 8 connection(s)
I20260812 14:42:24.023352 17759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:24.032485 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.024109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:49773 (local address 127.15.227.254:44061)
0812 14:42:24.024168 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.024170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.024378 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:42:24.025271 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.025273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.025283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.025343 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:24.025933 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.026468 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.027459 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.027641 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.027689 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.029931 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:24.029933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:24.029933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:24.029941 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:24.031365 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.031751 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.031752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.031752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.031801 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.031985 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.031985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.031985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.032084 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:24.032129 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.032357 (+ 228us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.032419 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:42:24.032676 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.024083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:44061 (local address 127.15.227.194:49773)
0812 14:42:24.024252 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:24.024296 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:24.025164 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:24.025398 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:24.025401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:24.025666 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:24.025867 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.025871 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.026543 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.026543 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.027390 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.027392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.027742 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.027744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.027857 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.028381 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:24.028390 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:24.029821 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:24.031451 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:24.031458 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:24.031460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:24.031689 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:24.031846 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:24.031847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:24.031847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:24.031943 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:24.032124 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:24.032127 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:24.032259 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 14:42:24.032337 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:42:24.032459 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260812 14:42:24.033164 18013 heartbeater.cc:347] Connected to a master server at 127.15.227.254:44061
I20260812 14:42:24.033264 18013 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:24.033443 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 17759
I20260812 14:42:24.033478 18013 heartbeater.cc:511] Master 127.15.227.254:44061 requested a full tablet report, sending...
I20260812 14:42:24.033533 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:24.033713 16271 external_mini_cluster.cc:1428] Setting key 977a8e910ba5d0508bcb16cefb6d4de5
I20260812 14:42:24.033867 17362 ts_manager.cc:194] Registered new tserver with Master: ebc81ad902bd44698ce8aed0218476e7 (127.15.227.194:42115)
I20260812 14:42:24.034521 17362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:49773
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:24.056926 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:44061
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:24.137015 18021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:24.137212 18021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:24.137243 18021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:24.138374 18021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:24.138411 18021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:24.138446 18021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:24.139899 18021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:44061
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:24.140131 18021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:24.140326 18021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:24.144910 18031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:24.145002 18021 server_base.cc:1034] running on GCE node
W20260812 14:42:24.144940 18032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:24.144970 18035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:24.145310 18021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:24.145520 18021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:24.146661 18021 hybrid_clock.cc:648] HybridClock initialized: now 1786545744146646 us; error 33 us; skew 500 ppm
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:24.148974 18021 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:24.149358 18021 webserver.cc:460] Webserver started at http://127.15.227.195:35501/ using document root <none> and password file <none>
I20260812 14:42:24.149554 18021 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:24.149626 18021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:24.149752 18021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:24.150770 18021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "16cd28203c3840ff807791db5eebf969"
format_stamp: "Formatted at 2026-08-12 14:42:24 on dist-test-slave-q9h9"
server_key: "baf18d7542a1e7d1c87536b5f9cdad21"
server_key_iv: "0793730cd9844d1189fc0fe66e120bea"
server_key_version: "encryptionkey@0"
I20260812 14:42:24.151078 18021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "16cd28203c3840ff807791db5eebf969"
format_stamp: "Formatted at 2026-08-12 14:42:24 on dist-test-slave-q9h9"
server_key: "baf18d7542a1e7d1c87536b5f9cdad21"
server_key_iv: "0793730cd9844d1189fc0fe66e120bea"
server_key_version: "encryptionkey@0"
I20260812 14:42:24.152297 18021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:42:24.152966 18046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:24.153163 18021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:24.153250 18021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "16cd28203c3840ff807791db5eebf969"
format_stamp: "Formatted at 2026-08-12 14:42:24 on dist-test-slave-q9h9"
server_key: "baf18d7542a1e7d1c87536b5f9cdad21"
server_key_iv: "0793730cd9844d1189fc0fe66e120bea"
server_key_version: "encryptionkey@0"
I20260812 14:42:24.153307 18021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:24.164671 18021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:24.166335 18021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:24.166456 18021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:24.166625 18021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:24.166870 18021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:24.166903 18021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:24.166924 18021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:24.166944 18021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:24.173206 18021 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:36779
I20260812 14:42:24.173265 18274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:36779 every 8 connection(s)
I20260812 14:42:24.173537 18021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545744, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:24.181059 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18021
I20260812 14:42:24.181164 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:24.181356 16271 external_mini_cluster.cc:1428] Setting key 90dba75f688bcdfbe25f1c9fd3e7870b
I20260812 14:42:24.182404 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.174205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:34107 (local address 127.15.227.254:44061)
0812 14:42:24.174260 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.174263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.174469 (+ 206us) server_negotiation.cc:407] Connection header received
0812 14:42:24.175519 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.175521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.175536 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.175601 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:24.176201 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.176713 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.177689 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.177897 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.177961 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.179930 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:24.179931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:24.179931 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:24.179938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:24.180987 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.181424 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.181425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.181425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.181491 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.181765 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.181766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.181766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.181901 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:24.181963 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.182267 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.182341 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:42:24.182524 18282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.174168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:44061 (local address 127.15.227.195:34107)
0812 14:42:24.174353 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:24.174390 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:24.175401 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:24.175669 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:24.175673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:24.175924 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:24.176103 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.176107 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.176803 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.176805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.177597 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.177600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.177973 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.177975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.178129 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.178546 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:24.178563 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:24.179835 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:24.181061 (+ 1226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:24.181067 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:24.181069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:24.181350 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:24.181558 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:24.181559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:24.181559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:24.181689 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:24.181961 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:24.181967 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:24.182126 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 14:42:24.182230 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 14:42:24.182362 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260812 14:42:24.182929 18276 heartbeater.cc:347] Connected to a master server at 127.15.227.254:44061
I20260812 14:42:24.183014 18276 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:24.183182 18276 heartbeater.cc:511] Master 127.15.227.254:44061 requested a full tablet report, sending...
I20260812 14:42:24.183547 17362 ts_manager.cc:194] Registered new tserver with Master: 16cd28203c3840ff807791db5eebf969 (127.15.227.195:36779)
I20260812 14:42:24.183931 17362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:34107
I20260812 14:42:24.192838 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:42:24.200260 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.194766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53834 (local address 127.15.227.254:44061)
0812 14:42:24.194822 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.194824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.194939 (+ 115us) server_negotiation.cc:407] Connection header received
0812 14:42:24.194979 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.194979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.194987 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.195038 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:24.195595 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.196124 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.196887 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.197109 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.197163 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.198250 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:24.198251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:24.198251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:24.198258 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:24.199257 (+ 999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.199549 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.199550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.199550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.199603 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.199765 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.199766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.199767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.199900 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:24.199986 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.200103 (+ 117us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.200177 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:42:24.202864 17362 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:24.204542 17362 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:24.210795 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.206983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:42115 (local address 127.0.0.1:41992)
0812 14:42:24.207150 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:24.207184 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:24.207247 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:24.207527 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:24.207530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:24.207531 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:24.207591 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.207593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.208528 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.208529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.209583 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.209585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.210505 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.210506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.210578 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.210591 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 14:42:24.210659 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:42:24.210711 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":51,"threads_started":1}
I20260812 14:42:24.210939 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.207187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37916 (local address 127.15.227.195:36779)
0812 14:42:24.207481 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.207485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.207643 (+ 158us) server_negotiation.cc:407] Connection header received
0812 14:42:24.207680 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.207682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.207695 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.207770 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:24.207872 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.208899 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.209991 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.210500 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.210567 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.210618 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.210837 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.210872 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":91,"threads_started":1}
I20260812 14:42:24.211078 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.206997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:36779 (local address 127.0.0.1:37916)
0812 14:42:24.207523 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:24.207534 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:24.207591 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:24.207778 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:24.207779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:24.207781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:24.207816 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.207817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.208998 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.208999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.209921 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.209923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.210651 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.210652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.210703 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.210706 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:24.210750 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:24.210785 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":58,"threads_started":1}
I20260812 14:42:24.211174 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.206984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36699 (local address 127.0.0.1:54834)
0812 14:42:24.207352 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:24.207361 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:24.207411 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:24.207670 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:24.207671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:24.207673 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:24.207726 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.207728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.209061 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.209062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.209973 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.209976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.210945 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.210946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.211005 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.211010 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:42:24.211056 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:42:24.211104 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260812 14:42:24.210804 18298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.207088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41992 (local address 127.15.227.194:42115)
0812 14:42:24.207343 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.207348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.207359 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:24.207401 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.207403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.207419 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.207520 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:24.207653 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.208443 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.209673 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.210420 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.210493 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.210544 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.210681 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.210723 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":101,"threads_started":1}
I20260812 14:42:24.211464 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.207239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.15.227.193:36699)
0812 14:42:24.207393 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.207397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.207407 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:24.207485 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.207487 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.207503 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.207581 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:24.207802 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.208891 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.210110 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.210786 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.210853 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.210906 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.211341 (+ 435us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.211385 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260812 14:42:24.211796 18136 tablet_service.cc:1467] Processing CreateTablet for tablet 276866b4c8e840dcbf0e7c0811a788ed (DEFAULT_TABLE table=test-table [id=f004e59b0a2c41578f09b82293e5a72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:24.212098 18136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 276866b4c8e840dcbf0e7c0811a788ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:24.211974 17871 tablet_service.cc:1467] Processing CreateTablet for tablet 276866b4c8e840dcbf0e7c0811a788ed (DEFAULT_TABLE table=test-table [id=f004e59b0a2c41578f09b82293e5a72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:24.212054 17619 tablet_service.cc:1467] Processing CreateTablet for tablet 276866b4c8e840dcbf0e7c0811a788ed (DEFAULT_TABLE table=test-table [id=f004e59b0a2c41578f09b82293e5a72e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:24.212324 17871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 276866b4c8e840dcbf0e7c0811a788ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:24.212347 17619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 276866b4c8e840dcbf0e7c0811a788ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:24.215080 18302 tablet_bootstrap.cc:492] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969: Bootstrap starting.
I20260812 14:42:24.215772 18303 tablet_bootstrap.cc:492] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7: Bootstrap starting.
I20260812 14:42:24.216229 18302 tablet_bootstrap.cc:654] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:24.216739 18304 tablet_bootstrap.cc:492] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8: Bootstrap starting.
I20260812 14:42:24.217370 18303 tablet_bootstrap.cc:654] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:24.217381 18302 log.cc:824] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:24.217900 18304 tablet_bootstrap.cc:654] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:24.218343 18302 tablet_bootstrap.cc:492] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969: No bootstrap required, opened a new log
I20260812 14:42:24.218425 18302 ts_tablet_manager.cc:1397] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:42:24.218569 18303 log.cc:824] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:24.219138 18304 log.cc:824] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:24.219601 18303 tablet_bootstrap.cc:492] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7: No bootstrap required, opened a new log
I20260812 14:42:24.219672 18303 ts_tablet_manager.cc:1397] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 14:42:24.220049 18304 tablet_bootstrap.cc:492] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8: No bootstrap required, opened a new log
I20260812 14:42:24.220111 18304 ts_tablet_manager.cc:1397] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:42:24.222456 18302 raft_consensus.cc:348] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.222685 18302 raft_consensus.cc:374] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:24.222738 18302 raft_consensus.cc:729] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16cd28203c3840ff807791db5eebf969, State: Initialized, Role: FOLLOWER
I20260812 14:42:24.222875 18302 consensus_queue.cc:260] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.223296 18276 heartbeater.cc:503] Master 127.15.227.254:44061 was elected leader, sending a full tablet report...
I20260812 14:42:24.223327 18302 ts_tablet_manager.cc:1428] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 14:42:24.223608 18304 raft_consensus.cc:348] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.223830 18304 raft_consensus.cc:374] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:24.223882 18304 raft_consensus.cc:729] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 322cba8524d54f6e9306a966b834bba8, State: Initialized, Role: FOLLOWER
I20260812 14:42:24.224011 18304 consensus_queue.cc:260] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.224014 18303 raft_consensus.cc:348] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.224229 18303 raft_consensus.cc:374] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:24.224313 18303 raft_consensus.cc:729] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebc81ad902bd44698ce8aed0218476e7, State: Initialized, Role: FOLLOWER
I20260812 14:42:24.224460 17751 heartbeater.cc:503] Master 127.15.227.254:44061 was elected leader, sending a full tablet report...
I20260812 14:42:24.224445 18303 consensus_queue.cc:260] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.224480 18304 ts_tablet_manager.cc:1428] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260812 14:42:24.224987 18303 ts_tablet_manager.cc:1428] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260812 14:42:24.225234 18013 heartbeater.cc:503] Master 127.15.227.254:44061 was elected leader, sending a full tablet report...
W20260812 14:42:24.273917 18014 tablet.cc:2369] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:42:24.372332 17754 tablet.cc:2369] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:42:24.424185 18280 tablet.cc:2369] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:24.545379 18319 raft_consensus.cc:482] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:24.545507 18319 raft_consensus.cc:504] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.546041 18319 leader_election.cc:288] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16cd28203c3840ff807791db5eebf969 (127.15.227.195:36779), 322cba8524d54f6e9306a966b834bba8 (127.15.227.193:36699)
I20260812 14:42:24.550207 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.546380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:58011 (local address 127.15.227.195:36779)
0812 14:42:24.546423 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.546425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.546435 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:24.546494 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.546495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.546505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.546570 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:24.546874 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.547589 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.549163 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.549784 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.549849 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.549858 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.550120 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.550155 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:24.550308 18325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.546231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:36779 (local address 127.15.227.194:58011)
0812 14:42:24.546367 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:24.546378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:24.546436 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:24.546617 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:24.546619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:24.546622 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:24.546786 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.546791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.547681 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.547682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.549076 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.549080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.549858 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.549859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.549979 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.549985 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 14:42:24.550107 (+ 122us) client_negotiation.cc:240] Negotiation successful
0812 14:42:24.550155 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":37,"thread_start_us":59,"threads_started":1}
I20260812 14:42:24.550369 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.546309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:40009 (local address 127.15.227.193:36699)
0812 14:42:24.546366 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.546368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.546438 (+ 70us) server_negotiation.cc:407] Connection header received
0812 14:42:24.546511 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.546513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.546522 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.546571 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:24.546975 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.547665 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.549432 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.549937 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.549987 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.549993 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.550266 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.550304 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:42:24.550581 18324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.546231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36699 (local address 127.15.227.194:40009)
0812 14:42:24.546367 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:24.546378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:24.546437 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:24.546617 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:24.546619 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:24.546623 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:24.546896 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.546899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.547745 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.547746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.549344 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:24.549347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.550044 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.550046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:24.550173 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.550178 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:42:24.550238 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:42:24.550275 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260812 14:42:24.550887 18180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276866b4c8e840dcbf0e7c0811a788ed" candidate_uuid: "ebc81ad902bd44698ce8aed0218476e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cd28203c3840ff807791db5eebf969" is_pre_election: true
I20260812 14:42:24.551060 18180 raft_consensus.cc:2454] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc81ad902bd44698ce8aed0218476e7 in term 0.
I20260812 14:42:24.551002 17656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276866b4c8e840dcbf0e7c0811a788ed" candidate_uuid: "ebc81ad902bd44698ce8aed0218476e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "322cba8524d54f6e9306a966b834bba8" is_pre_election: true
I20260812 14:42:24.551177 17656 raft_consensus.cc:2454] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebc81ad902bd44698ce8aed0218476e7 in term 0.
I20260812 14:42:24.551318 17801 leader_election.cc:302] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cd28203c3840ff807791db5eebf969, ebc81ad902bd44698ce8aed0218476e7; no voters:
I20260812 14:42:24.551469 18319 raft_consensus.cc:2781] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:24.551537 18319 raft_consensus.cc:482] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:24.551576 18319 raft_consensus.cc:3037] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:24.552575 18319 raft_consensus.cc:504] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.552716 18319 leader_election.cc:288] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [CANDIDATE]: Term 1 election: Requested vote from peers 16cd28203c3840ff807791db5eebf969 (127.15.227.195:36779), 322cba8524d54f6e9306a966b834bba8 (127.15.227.193:36699)
I20260812 14:42:24.552855 18180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276866b4c8e840dcbf0e7c0811a788ed" candidate_uuid: "ebc81ad902bd44698ce8aed0218476e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16cd28203c3840ff807791db5eebf969"
I20260812 14:42:24.552933 18180 raft_consensus.cc:3037] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:24.552989 17656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "276866b4c8e840dcbf0e7c0811a788ed" candidate_uuid: "ebc81ad902bd44698ce8aed0218476e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "322cba8524d54f6e9306a966b834bba8"
I20260812 14:42:24.553085 17656 raft_consensus.cc:3037] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:24.554314 17656 raft_consensus.cc:2454] T 276866b4c8e840dcbf0e7c0811a788ed P 322cba8524d54f6e9306a966b834bba8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc81ad902bd44698ce8aed0218476e7 in term 1.
I20260812 14:42:24.554358 18180 raft_consensus.cc:2454] T 276866b4c8e840dcbf0e7c0811a788ed P 16cd28203c3840ff807791db5eebf969 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebc81ad902bd44698ce8aed0218476e7 in term 1.
I20260812 14:42:24.554487 17801 leader_election.cc:302] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16cd28203c3840ff807791db5eebf969, ebc81ad902bd44698ce8aed0218476e7; no voters:
I20260812 14:42:24.554584 18319 raft_consensus.cc:2781] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:24.554804 18319 raft_consensus.cc:686] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [term 1 LEADER]: Becoming Leader. State: Replica: ebc81ad902bd44698ce8aed0218476e7, State: Running, Role: LEADER
I20260812 14:42:24.554888 18319 consensus_queue.cc:237] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } }
I20260812 14:42:24.555845 17362 catalog_manager.cc:5495] T 276866b4c8e840dcbf0e7c0811a788ed P ebc81ad902bd44698ce8aed0218476e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebc81ad902bd44698ce8aed0218476e7 (127.15.227.194). New cstate: current_term: 1 leader_uuid: "ebc81ad902bd44698ce8aed0218476e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebc81ad902bd44698ce8aed0218476e7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16cd28203c3840ff807791db5eebf969" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 36779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "322cba8524d54f6e9306a966b834bba8" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36699 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:24.567040 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.559899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54846 (local address 127.15.227.193:36699)
0812 14:42:24.559965 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.559967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.560138 (+ 171us) server_negotiation.cc:407] Connection header received
0812 14:42:24.560202 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.560203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.560213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.560265 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:24.560827 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.561267 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.561912 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.562159 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.562231 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.564490 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:24.564499 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:24.564499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:24.564506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:24.566110 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.566400 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.566402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.566403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.566494 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.566653 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.566654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.566654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.566779 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:24.566873 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.566926 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.566980 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545744, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545744, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:24.591384 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.585302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54862 (local address 127.15.227.193:36699)
0812 14:42:24.585356 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.585358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.585417 (+ 59us) server_negotiation.cc:407] Connection header received
0812 14:42:24.585460 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.585460 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.585469 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.585514 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:24.586108 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.586558 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.587208 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.587382 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.587421 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.589392 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:24.589394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:24.589394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:24.589401 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:24.590572 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.590835 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.590835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.590836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.590878 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.590994 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.590995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.590995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.591066 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:24.591145 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.591268 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.591322 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[17287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:24.612470 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:24.606305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.15.227.193:36699)
0812 14:42:24.606371 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:24.606372 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:24.606440 (+ 68us) server_negotiation.cc:407] Connection header received
0812 14:42:24.606498 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:24.606499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:24.606507 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:24.606551 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:24.607040 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.607461 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.608138 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:24.608301 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:24.608339 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:24.610517 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:24.610518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:24.610519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:24.610524 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:24.611634 (+ 1110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.611955 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.611955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.611956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.611996 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:24.612140 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:24.612140 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:24.612140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:24.612212 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:24.612284 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:24.612361 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:42:24.612411 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 14:42:24.612943 17580 tablet_service.cc:2908] Rejecting scan request for tablet 276866b4c8e840dcbf0e7c0811a788ed: Uninitialized: clean time has not yet been initialized
I20260812 14:42:24.613776 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 17484
I20260812 14:42:24.676434 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 17759
I20260812 14:42:24.684115 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18021
I20260812 14:42:24.694916 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 17310
2026-08-12T14:42:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2423 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[18351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[18351](info): set up 2 sockets
Aug 12 14:42:24 dist-test-slave-q9h9 krb5kdc[18351](info): commencing operation
krb5kdc: starting...
W20260812 14:42:25.893795 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:25 dist-test-slave-q9h9 krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:26.009126 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:39443
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:39443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:26.091106 18372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:26.091297 18372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:26.091327 18372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:26.092432 18372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:26.092473 18372 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:42:26.092490 18372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:26.092506 18372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:26.092522 18372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:26.092545 18372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:26.093983 18372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:39443
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:39443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:26.094234 18372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:26.094434 18372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:26.099001 18385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:26.099045 18386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:26.099098 18372 server_base.cc:1034] running on GCE node
W20260812 14:42:26.099021 18390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:26.099587 18372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:26.099821 18372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:26.100950 18372 hybrid_clock.cc:648] HybridClock initialized: now 1786545746100937 us; error 30 us; skew 500 ppm
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:26.103291 18372 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:26.103708 18372 webserver.cc:460] Webserver started at http://127.15.227.254:39937/ using document root <none> and password file <none>
I20260812 14:42:26.103878 18372 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:26.103917 18372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:26.104004 18372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:26.104940 18372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "1e4031bacc2349628214f6d8f83e4b8a"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "4613a2dfac17c94c9fc1c90a7d20bb6e"
server_key_iv: "7a28c1b3fd886ceb1b0201b6d7742c94"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.105226 18372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "1e4031bacc2349628214f6d8f83e4b8a"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "4613a2dfac17c94c9fc1c90a7d20bb6e"
server_key_iv: "7a28c1b3fd886ceb1b0201b6d7742c94"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.106525 18372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:26.107149 18403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.107285 18372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:26.107362 18372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "1e4031bacc2349628214f6d8f83e4b8a"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "4613a2dfac17c94c9fc1c90a7d20bb6e"
server_key_iv: "7a28c1b3fd886ceb1b0201b6d7742c94"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.107425 18372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:26.126384 18372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:26.128124 18372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:26.128249 18372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:26.132896 18372 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:39443
I20260812 14:42:26.132972 18514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:39443 every 8 connection(s)
I20260812 14:42:26.133260 18372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:26.133912 18515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:26.134011 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18372
I20260812 14:42:26.134125 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:26.134289 16271 external_mini_cluster.cc:1428] Setting key 6c3988f5863de366b5ebe320570a9144
I20260812 14:42:26.136795 18515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a: Bootstrap starting.
I20260812 14:42:26.137378 18515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:26.137650 18515 log.cc:824] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:26.138278 18515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a: No bootstrap required, opened a new log
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:26.140581 18515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4031bacc2349628214f6d8f83e4b8a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39443 } }
I20260812 14:42:26.140717 18515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:26.140739 18515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4031bacc2349628214f6d8f83e4b8a, State: Initialized, Role: FOLLOWER
I20260812 14:42:26.140859 18515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4031bacc2349628214f6d8f83e4b8a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39443 } }
I20260812 14:42:26.140919 18515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:26.140942 18515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:26.140987 18515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:26.141577 18515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4031bacc2349628214f6d8f83e4b8a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39443 } }
I20260812 14:42:26.141696 18515 leader_election.cc:302] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e4031bacc2349628214f6d8f83e4b8a; no voters:
I20260812 14:42:26.141858 18515 leader_election.cc:288] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:26.141889 18520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:26.141994 18520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [term 1 LEADER]: Becoming Leader. State: Replica: 1e4031bacc2349628214f6d8f83e4b8a, State: Running, Role: LEADER
I20260812 14:42:26.142138 18515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:26.142166 18520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4031bacc2349628214f6d8f83e4b8a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39443 } }
I20260812 14:42:26.142663 18521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e4031bacc2349628214f6d8f83e4b8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4031bacc2349628214f6d8f83e4b8a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39443 } } }
I20260812 14:42:26.142669 18523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e4031bacc2349628214f6d8f83e4b8a. Latest consensus state: current_term: 1 leader_uuid: "1e4031bacc2349628214f6d8f83e4b8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4031bacc2349628214f6d8f83e4b8a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39443 } } }
I20260812 14:42:26.142752 18521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:26.142755 18523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:26.143267 18534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:26.143800 18534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:26.144585 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.134752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46058 (local address 127.15.227.254:39443)
0812 14:42:26.134938 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.134945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.134969 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:42:26.135389 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.135396 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.135440 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.135558 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.136161 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.137064 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.137829 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.138279 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.138372 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.140732 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.140736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.140736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.140749 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.142488 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.142960 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.142963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.142964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.143068 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.143450 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.143451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.143452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.143741 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.143880 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.144259 (+ 379us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.144393 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
I20260812 14:42:26.145685 18534 catalog_manager.cc:1347] Generated new cluster ID: a499d3cd5350461eabd6c1bdec606853
I20260812 14:42:26.145735 18534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:26.154935 18534 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:26.155385 18534 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:26.162992 18534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a: Generated new TSK 0
I20260812 14:42:26.163139 18534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:26.196058 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:39443
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:26.277612 18555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:26.277796 18555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:26.277817 18555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:26.278946 18555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:26.278985 18555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:26.279004 18555 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:26.279022 18555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:26.280426 18555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:26.280625 18555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:26.280843 18555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:26.285465 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:26.285459 18567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:26.285459 18569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:26.285516 18555 server_base.cc:1034] running on GCE node
I20260812 14:42:26.285845 18555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:26.286079 18555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:26.287215 18555 hybrid_clock.cc:648] HybridClock initialized: now 1786545746287202 us; error 31 us; skew 500 ppm
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:26.289414 18555 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:26.289804 18555 webserver.cc:460] Webserver started at http://127.15.227.193:46163/ using document root <none> and password file <none>
I20260812 14:42:26.290050 18555 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:26.290109 18555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:26.290234 18555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:26.291287 18555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "81d68200041b4837b5ede77784a09e4f"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "2e6b2e9056f865a65de059167fceec4b"
server_key_iv: "469aad205a6a6ee245a1c0ff332272c2"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.291647 18555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "81d68200041b4837b5ede77784a09e4f"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "2e6b2e9056f865a65de059167fceec4b"
server_key_iv: "469aad205a6a6ee245a1c0ff332272c2"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.292821 18555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:26.293488 18580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.293628 18555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:26.293699 18555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "81d68200041b4837b5ede77784a09e4f"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "2e6b2e9056f865a65de059167fceec4b"
server_key_iv: "469aad205a6a6ee245a1c0ff332272c2"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.293766 18555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:26.307042 18555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:26.308547 18555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:26.308665 18555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:26.309090 18555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:26.309125 18555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.309150 18555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:26.309167 18555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:26.315922 18555 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:35417
I20260812 14:42:26.315984 18799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:35417 every 8 connection(s)
I20260812 14:42:26.316287 18555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:26.317836 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.309632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:44469 (local address 127.15.227.254:39443)
0812 14:42:26.309675 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.309677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.309865 (+ 188us) server_negotiation.cc:407] Connection header received
0812 14:42:26.310371 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.310373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.310385 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.310456 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.311140 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.311668 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.312629 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.312941 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.313010 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.315330 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.315331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.315332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.315339 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.316825 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.317177 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.317178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.317178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.317224 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.317390 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.317391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.317391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.317479 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.317520 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.317712 (+ 192us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.317774 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:26.317978 18593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.309596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:39443 (local address 127.15.227.193:44469)
0812 14:42:26.309760 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.309802 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.310279 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.310521 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.310524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.310851 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:26.311048 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.311053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.311752 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.311753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.312546 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.312548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.313033 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.313035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.313197 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.313709 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:26.313719 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:26.315247 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:26.316899 (+ 1652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.316903 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.316905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.317118 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.317269 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.317270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.317270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.317351 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.317519 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:26.317522 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:26.317624 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.317695 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.317803 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260812 14:42:26.318432 18800 heartbeater.cc:347] Connected to a master server at 127.15.227.254:39443
I20260812 14:42:26.318519 18800 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:26.318681 18800 heartbeater.cc:511] Master 127.15.227.254:39443 requested a full tablet report, sending...
I20260812 14:42:26.319157 18436 ts_manager.cc:194] Registered new tserver with Master: 81d68200041b4837b5ede77784a09e4f (127.15.227.193:35417)
I20260812 14:42:26.319896 18436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:44469
I20260812 14:42:26.321374 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18555
I20260812 14:42:26.321465 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:26.321614 16271 external_mini_cluster.cc:1428] Setting key 044104ba7cd24f8c77ca733c55e4c661
I20260812 14:42:26.324404 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.319432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:56393 (local address 127.15.227.254:39443)
0812 14:42:26.319469 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.319471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.319593 (+ 122us) server_negotiation.cc:407] Connection header received
0812 14:42:26.319640 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.319641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.319651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.319702 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.319958 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.320496 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.321166 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.321339 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.321389 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.322272 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.322273 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.322273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.322279 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.323502 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.323791 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.323792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.323792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.323855 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.324025 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.324025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.324025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.324115 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.324158 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.324296 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.324350 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:26.324452 18818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.319372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:39443 (local address 127.15.227.193:56393)
0812 14:42:26.319528 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.319536 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.319592 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.319705 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.319705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.319856 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:26.319900 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.319901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.320594 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.320595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.321105 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.321107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.321401 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.321402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.321481 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.321872 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:26.321877 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:26.322194 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:26.323571 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.323572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.323573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.323721 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.323915 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.323916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.323916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.323978 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.324161 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:26.324162 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:26.324205 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.324279 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.324368 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:26.344403 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:39443
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:26.422446 18824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:26.422624 18824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:26.422655 18824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:26.423763 18824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:26.423802 18824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:26.423830 18824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:26.423851 18824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:26.425217 18824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:26.425426 18824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:26.425619 18824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:26.430215 18837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:26.430231 18842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:26.430297 18838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:26.430459 18824 server_base.cc:1034] running on GCE node
I20260812 14:42:26.430653 18824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:26.430871 18824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:26.432088 18824 hybrid_clock.cc:648] HybridClock initialized: now 1786545746432069 us; error 28 us; skew 500 ppm
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:26.434396 18824 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:26.434769 18824 webserver.cc:460] Webserver started at http://127.15.227.194:40597/ using document root <none> and password file <none>
I20260812 14:42:26.434962 18824 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:26.435007 18824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:26.435127 18824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:26.436061 18824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "227484a3577b4c87883349a15c4c6fb1"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "b295a34a206efec0e094cb3368a6f100"
server_key_iv: "4e517e47496263e706ee4b9a3735f3cf"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.436370 18824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "227484a3577b4c87883349a15c4c6fb1"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "b295a34a206efec0e094cb3368a6f100"
server_key_iv: "4e517e47496263e706ee4b9a3735f3cf"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.437558 18824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:42:26.438277 18856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.438413 18824 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.438503 18824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "227484a3577b4c87883349a15c4c6fb1"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "b295a34a206efec0e094cb3368a6f100"
server_key_iv: "4e517e47496263e706ee4b9a3735f3cf"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.438557 18824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:26.451135 18824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:26.452888 18824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:26.453019 18824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:26.453473 18824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:26.453514 18824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.453542 18824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:26.453559 18824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:26.460320 18824 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:37111
I20260812 14:42:26.460387 19084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:37111 every 8 connection(s)
I20260812 14:42:26.460705 18824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:26.462703 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.453904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:38681 (local address 127.15.227.254:39443)
0812 14:42:26.453965 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.453967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.454200 (+ 233us) server_negotiation.cc:407] Connection header received
0812 14:42:26.454814 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.454815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.454827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.454889 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.455709 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.456341 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.457308 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.457515 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.457571 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.460201 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.460202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.460202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.460210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.461617 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.461953 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.461953 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.461954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.461999 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.462251 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.462252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.462252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.462351 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.462396 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.462585 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.462643 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:42:26.462841 18872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.453872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:39443 (local address 127.15.227.194:38681)
0812 14:42:26.454064 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.454112 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.454704 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.454965 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.454969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.455323 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:26.455595 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.455602 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.456437 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.456438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.457225 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.457228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.457583 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.457584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.457728 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.458259 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:26.458269 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:26.460103 (+ 1834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:26.461693 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.461696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.461698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.461900 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.462084 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.462085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.462085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.462202 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.462393 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:26.462396 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:26.462498 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.462567 (+ 69us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.462679 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260812 14:42:26.463295 19085 heartbeater.cc:347] Connected to a master server at 127.15.227.254:39443
I20260812 14:42:26.463368 19085 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:26.463531 19085 heartbeater.cc:511] Master 127.15.227.254:39443 requested a full tablet report, sending...
I20260812 14:42:26.463905 18434 ts_manager.cc:194] Registered new tserver with Master: 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194:37111)
I20260812 14:42:26.464363 18434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:38681
I20260812 14:42:26.468376 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18824
I20260812 14:42:26.468461 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:26.468662 16271 external_mini_cluster.cc:1428] Setting key 98bf89600a44d4eacabee119428cdb2a
I20260812 14:42:26.468672 19091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.464161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:39443 (local address 127.15.227.194:38255)
0812 14:42:26.464271 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.464282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.464338 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.464468 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.464469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.464621 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:26.464669 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.464671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.465266 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.465267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.465772 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.465774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.466122 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.466123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.466198 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.466491 (+ 293us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:26.466495 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:26.466728 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:26.467774 (+ 1046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.467774 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.467775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.467922 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.468082 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.468083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.468083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.468139 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.468385 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:26.468386 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:26.468432 (+ 46us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.468523 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.468605 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
I20260812 14:42:26.468732 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.464236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:38255 (local address 127.15.227.254:39443)
0812 14:42:26.464273 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.464275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.464326 (+ 51us) server_negotiation.cc:407] Connection header received
0812 14:42:26.464386 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.464387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.464394 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.464441 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.464732 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.465182 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.465838 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.466060 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.466121 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.466806 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.466807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.466808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.466814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.467702 (+ 888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.467979 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.467980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.467980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.468031 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.468188 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.468189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.468189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.468328 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.468384 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.468580 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.468658 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:26.490123 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:39443
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:26.568087 19096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:26.568269 19096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:26.568300 19096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:26.569391 19096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:26.569430 19096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:26.569456 19096 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:26.569479 19096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:26.570892 19096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:39443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:26.571113 19096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:26.571305 19096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:26.575805 19107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:26.575814 19104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:26.576076 19096 server_base.cc:1034] running on GCE node
W20260812 14:42:26.575858 19103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:26.576272 19096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:26.576457 19096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:26.577591 19096 hybrid_clock.cc:648] HybridClock initialized: now 1786545746577580 us; error 28 us; skew 500 ppm
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:26.579916 19096 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:26.580268 19096 webserver.cc:460] Webserver started at http://127.15.227.195:34109/ using document root <none> and password file <none>
I20260812 14:42:26.580438 19096 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:26.580475 19096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:26.580559 19096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:26.581451 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "f50b3ace997448ac96aeda06caeb8c33"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "3af25da1eb3660f7295c644535ee03b9"
server_key_iv: "2a58b047fa3b27d20d2afec1e3559026"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.581733 19096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "f50b3ace997448ac96aeda06caeb8c33"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "3af25da1eb3660f7295c644535ee03b9"
server_key_iv: "2a58b047fa3b27d20d2afec1e3559026"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.582918 19096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260812 14:42:26.583500 19119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.583635 19096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.583710 19096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "f50b3ace997448ac96aeda06caeb8c33"
format_stamp: "Formatted at 2026-08-12 14:42:26 on dist-test-slave-q9h9"
server_key: "3af25da1eb3660f7295c644535ee03b9"
server_key_iv: "2a58b047fa3b27d20d2afec1e3559026"
server_key_version: "encryptionkey@0"
I20260812 14:42:26.583778 19096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:26.610890 19096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:26.612593 19096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:26.612715 19096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:26.613150 19096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:26.613188 19096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.613212 19096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:26.613229 19096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:26.619271 19096 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35579
I20260812 14:42:26.619349 19336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35579 every 8 connection(s)
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:26.619654 19096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:42:26.622246 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.613531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:37447 (local address 127.15.227.254:39443)
0812 14:42:26.613572 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.613574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.613780 (+ 206us) server_negotiation.cc:407] Connection header received
0812 14:42:26.614265 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.614266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.614278 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.614328 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.615087 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.615822 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.617214 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.617514 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.617570 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.619935 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.619936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.619937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.619945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.621111 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.621470 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.621471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.621471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.621532 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.621714 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.621714 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.621714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.621810 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.621857 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.622106 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.622182 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:42:26.622324 19137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.613535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:39443 (local address 127.15.227.195:37447)
0812 14:42:26.613676 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.613707 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.614172 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.614390 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.614394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.614721 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:26.614981 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.614987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.615917 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.615918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.617132 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.617136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.617577 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.617578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.617706 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.618168 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:26.618179 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:26.619853 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:26.621177 (+ 1324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.621181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.621183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.621390 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.621586 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.621587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.621587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.621671 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.621852 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:26.621855 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:26.621950 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.622053 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.622173 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260812 14:42:26.622784 19338 heartbeater.cc:347] Connected to a master server at 127.15.227.254:39443
I20260812 14:42:26.622864 19338 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:26.623030 19338 heartbeater.cc:511] Master 127.15.227.254:39443 requested a full tablet report, sending...
I20260812 14:42:26.623394 18434 ts_manager.cc:194] Registered new tserver with Master: f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579)
I20260812 14:42:26.623847 18434 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:37447
I20260812 14:42:26.624212 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 19096
I20260812 14:42:26.624325 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:26.624495 16271 external_mini_cluster.cc:1428] Setting key 10d8778bc11c4add03764e6f1fc42993
I20260812 14:42:26.625268 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:42:26.629230 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.623792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:37709 (local address 127.15.227.254:39443)
0812 14:42:26.623978 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.623980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.623986 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:42:26.624004 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.624004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.624013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.624061 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.624442 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.625005 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.625888 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.626124 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.626176 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.626965 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.626967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.626967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.626974 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.628330 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.628617 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.628618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.628618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.628671 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.628838 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.628839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.628839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.628921 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.628964 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.629117 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.629178 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260812 14:42:26.629243 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.623735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:39443 (local address 127.15.227.195:37709)
0812 14:42:26.623854 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.623863 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.623908 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.624074 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.624075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.624258 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:26.624335 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.624337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.625081 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.625082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.625815 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.625818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.626187 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.626188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.626257 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.626583 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:26.626587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:26.626884 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:26.628412 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.628413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.628413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.628558 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.628726 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.628727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.628727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.628791 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.628966 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:26.628967 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:26.629009 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.629071 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.629153 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:26.647383 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.641455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46062 (local address 127.15.227.254:39443)
0812 14:42:26.641504 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.641505 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.641531 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:42:26.641581 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.641581 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.641589 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.641632 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.642261 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.642724 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.643466 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.643636 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.643686 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.645594 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.645596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.645596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.645602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.646617 (+ 1015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.646885 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.646886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.646886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.646928 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.647046 (+ 118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.647046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.647046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.647124 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.647184 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.647266 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.647316 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:42:26.648743 18434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:26.650334 18434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:26.656235 19368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.652312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:37111 (local address 127.0.0.1:53158)
0812 14:42:26.652401 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.652410 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.652450 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.652840 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.652841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.652843 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:26.652897 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.652899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.654191 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.654192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.655288 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.655291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.655933 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.655934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.656018 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.656033 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.656117 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.656169 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":55,"threads_started":1}
I20260812 14:42:26.657066 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.652426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53158 (local address 127.15.227.194:37111)
0812 14:42:26.652686 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.652690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.652700 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:26.652736 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.652738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.652753 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.652828 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:26.652976 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.654115 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.655361 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.655853 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.655920 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.655975 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.656926 (+ 951us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.656966 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":80,"threads_started":1}
I20260812 14:42:26.657074 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.652591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55672 (local address 127.15.227.193:35417)
0812 14:42:26.652781 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.652785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.652795 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:26.652831 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.652833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.652846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.653046 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:26.653113 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.653851 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.656012 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.656735 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.656796 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.656843 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.656943 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.656980 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260812 14:42:26.657087 19371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.652450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:35417 (local address 127.0.0.1:55672)
0812 14:42:26.652533 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.652540 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.652574 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.652952 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.652952 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.652954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:26.652992 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.652993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.653926 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.653927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.655288 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.655291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.656813 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.656814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.656879 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.656883 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.656932 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.656970 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":41,"threads_started":1}
I20260812 14:42:26.657094 19366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.652139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35579 (local address 127.0.0.1:43870)
0812 14:42:26.652252 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.652292 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.652357 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.652817 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.652820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.652822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:26.652884 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.652885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.655383 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.655384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.656047 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.656048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.656841 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.656842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.656906 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.656910 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.656960 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.657001 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
I20260812 14:42:26.657265 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.652240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43870 (local address 127.15.227.195:35579)
0812 14:42:26.652608 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.652612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.652621 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:26.652650 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.652651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.652664 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.652738 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:26.652967 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.654116 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.656124 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.656756 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.656827 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.656880 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.657138 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.657181 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":84,"threads_started":1}
I20260812 14:42:26.657811 18676 tablet_service.cc:1467] Processing CreateTablet for tablet a3c2adc2e6994ea3a7611c8dc5c07c33 (DEFAULT_TABLE table=test-table [id=16ea3d83b81344aea28e6a1f6cbdd1b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:26.657820 19219 tablet_service.cc:1467] Processing CreateTablet for tablet a3c2adc2e6994ea3a7611c8dc5c07c33 (DEFAULT_TABLE table=test-table [id=16ea3d83b81344aea28e6a1f6cbdd1b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:26.658157 18676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c2adc2e6994ea3a7611c8dc5c07c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:26.658213 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c2adc2e6994ea3a7611c8dc5c07c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:26.658159 18961 tablet_service.cc:1467] Processing CreateTablet for tablet a3c2adc2e6994ea3a7611c8dc5c07c33 (DEFAULT_TABLE table=test-table [id=16ea3d83b81344aea28e6a1f6cbdd1b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:26.658478 18961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c2adc2e6994ea3a7611c8dc5c07c33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:26.660876 19375 tablet_bootstrap.cc:492] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Bootstrap starting.
I20260812 14:42:26.661406 19377 tablet_bootstrap.cc:492] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Bootstrap starting.
I20260812 14:42:26.661548 19375 tablet_bootstrap.cc:654] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:26.661842 19378 tablet_bootstrap.cc:492] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Bootstrap starting.
I20260812 14:42:26.662225 19375 log.cc:824] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:26.662530 19378 tablet_bootstrap.cc:654] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:26.662581 19377 tablet_bootstrap.cc:654] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:26.662883 19378 log.cc:824] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:26.662886 19377 log.cc:824] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:26.663111 19375 tablet_bootstrap.cc:492] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: No bootstrap required, opened a new log
I20260812 14:42:26.663198 19375 ts_tablet_manager.cc:1397] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:42:26.663561 19377 tablet_bootstrap.cc:492] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: No bootstrap required, opened a new log
I20260812 14:42:26.663617 19377 ts_tablet_manager.cc:1397] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:42:26.664640 19375 raft_consensus.cc:348] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.664767 19375 raft_consensus.cc:374] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:26.664788 19375 raft_consensus.cc:729] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d68200041b4837b5ede77784a09e4f, State: Initialized, Role: FOLLOWER
I20260812 14:42:26.664885 19375 consensus_queue.cc:260] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.665127 19375 ts_tablet_manager.cc:1428] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:42:26.665180 18800 heartbeater.cc:503] Master 127.15.227.254:39443 was elected leader, sending a full tablet report...
I20260812 14:42:26.665108 19377 raft_consensus.cc:348] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.665251 19377 raft_consensus.cc:374] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:26.665285 19377 raft_consensus.cc:729] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50b3ace997448ac96aeda06caeb8c33, State: Initialized, Role: FOLLOWER
I20260812 14:42:26.665400 19377 consensus_queue.cc:260] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.665639 19377 ts_tablet_manager.cc:1428] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:42:26.665679 19378 tablet_bootstrap.cc:492] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: No bootstrap required, opened a new log
I20260812 14:42:26.665724 19378 ts_tablet_manager.cc:1397] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:42:26.665782 19338 heartbeater.cc:503] Master 127.15.227.254:39443 was elected leader, sending a full tablet report...
I20260812 14:42:26.667328 19378 raft_consensus.cc:348] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.667443 19378 raft_consensus.cc:374] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:26.667462 19378 raft_consensus.cc:729] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227484a3577b4c87883349a15c4c6fb1, State: Initialized, Role: FOLLOWER
I20260812 14:42:26.667563 19378 consensus_queue.cc:260] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.667773 19378 ts_tablet_manager.cc:1428] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:42:26.667806 19085 heartbeater.cc:503] Master 127.15.227.254:39443 was elected leader, sending a full tablet report...
W20260812 14:42:26.711249 19086 tablet.cc:2369] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:42:26.816978 18801 tablet.cc:2369] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:26.853579 19386 raft_consensus.cc:482] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:26.853706 19386 raft_consensus.cc:504] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.854112 19386 leader_election.cc:288] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579), 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194:37111)
I20260812 14:42:26.856926 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.854548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:49237 (local address 127.15.227.194:37111)
0812 14:42:26.854621 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.854623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.854631 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:26.854657 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.854657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.854665 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.854711 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:26.854860 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.855315 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.856275 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.856629 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.856690 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.856699 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.856829 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.856863 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:42:26.857003 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.854385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46065 (local address 127.15.227.195:35579)
0812 14:42:26.854438 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.854440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.854454 (+ 14us) server_negotiation.cc:407] Connection header received
0812 14:42:26.854513 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.854514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.854524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.854595 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:26.854813 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.855304 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.856278 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.856696 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.856760 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.856769 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.856903 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.856939 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:42:26.856942 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.854458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:37111 (local address 127.15.227.193:49237)
0812 14:42:26.854553 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.854561 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.854600 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.854719 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.854720 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.854722 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:26.854800 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.854802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.855382 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.855382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.856187 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.856189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.856696 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.856697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.856765 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.856769 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.856813 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.856853 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":28,"thread_start_us":56,"threads_started":1}
I20260812 14:42:26.857187 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.854277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35579 (local address 127.15.227.193:46065)
0812 14:42:26.854387 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.854399 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.854453 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.854663 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.854665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.854667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:26.854730 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.854732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.855378 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.855379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.856189 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.856192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.856769 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.856770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.856841 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.856845 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.856892 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.856936 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":29,"thread_start_us":67,"threads_started":1}
I20260812 14:42:26.857486 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c2adc2e6994ea3a7611c8dc5c07c33" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227484a3577b4c87883349a15c4c6fb1" is_pre_election: true
I20260812 14:42:26.857497 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c2adc2e6994ea3a7611c8dc5c07c33" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50b3ace997448ac96aeda06caeb8c33" is_pre_election: true
I20260812 14:42:26.857609 18997 raft_consensus.cc:2454] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 0.
I20260812 14:42:26.857618 19254 raft_consensus.cc:2454] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 0.
I20260812 14:42:26.857808 18584 leader_election.cc:302] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81d68200041b4837b5ede77784a09e4f, f50b3ace997448ac96aeda06caeb8c33; no voters:
I20260812 14:42:26.857942 19386 raft_consensus.cc:2781] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:26.858000 19386 raft_consensus.cc:482] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:26.858058 19386 raft_consensus.cc:3037] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:26.858737 19386 raft_consensus.cc:504] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.858839 19386 leader_election.cc:288] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 election: Requested vote from peers f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579), 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194:37111)
I20260812 14:42:26.859004 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c2adc2e6994ea3a7611c8dc5c07c33" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50b3ace997448ac96aeda06caeb8c33"
I20260812 14:42:26.859033 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c2adc2e6994ea3a7611c8dc5c07c33" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227484a3577b4c87883349a15c4c6fb1"
I20260812 14:42:26.859084 19254 raft_consensus.cc:3037] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:26.859091 18997 raft_consensus.cc:3037] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:26.859912 18997 raft_consensus.cc:2454] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 1.
I20260812 14:42:26.859912 19254 raft_consensus.cc:2454] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 1.
I20260812 14:42:26.860060 18584 leader_election.cc:302] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81d68200041b4837b5ede77784a09e4f, f50b3ace997448ac96aeda06caeb8c33; no voters:
I20260812 14:42:26.860162 19386 raft_consensus.cc:2781] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:26.860260 19386 raft_consensus.cc:686] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 1 LEADER]: Becoming Leader. State: Replica: 81d68200041b4837b5ede77784a09e4f, State: Running, Role: LEADER
I20260812 14:42:26.860333 19386 consensus_queue.cc:237] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } }
I20260812 14:42:26.861068 18434 catalog_manager.cc:5495] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 81d68200041b4837b5ede77784a09e4f (127.15.227.193). New cstate: current_term: 1 leader_uuid: "81d68200041b4837b5ede77784a09e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } health_report { overall_health: HEALTHY } } }
W20260812 14:42:26.870183 19341 tablet.cc:2369] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:26.883618 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.881330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55674 (local address 127.15.227.193:35417)
0812 14:42:26.881380 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.881382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.881390 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:26.881470 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.881471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.881486 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.881542 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:26.881758 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.882260 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.882950 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.883125 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.883168 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.883371 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:26.883455 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:26.883504 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.883539 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.883554 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:42:26.885483 19254 raft_consensus.cc:1261] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Refusing update from remote peer 81d68200041b4837b5ede77784a09e4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:26.885774 19386 consensus_queue.cc:1035] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:26.887320 18997 raft_consensus.cc:1261] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Refusing update from remote peer 81d68200041b4837b5ede77784a09e4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:26.887604 19386 consensus_queue.cc:1035] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:26.890281 19419 mvcc.cc:205] Tried to move back new op lower bound from 7317691379240329216 to 7317691379140501504. Current Snapshot: MvccSnapshot[applied={T|T < 7317691379240329216}]
I20260812 14:42:26.892338 18434 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:26.892452 18434 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:26.893260 18434 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a: Sending DeleteTablet for 3 replicas of tablet a3c2adc2e6994ea3a7611c8dc5c07c33
I20260812 14:42:26.893600 19219 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c2adc2e6994ea3a7611c8dc5c07c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:26 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:43870
I20260812 14:42:26.893600 18961 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c2adc2e6994ea3a7611c8dc5c07c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:26 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:53158
I20260812 14:42:26.893864 18676 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c2adc2e6994ea3a7611c8dc5c07c33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:26 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55672
I20260812 14:42:26.893909 18434 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:26.894074 18434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:26.894227 19424 tablet_replica.cc:330] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: stopping tablet replica
I20260812 14:42:26.894317 19424 raft_consensus.cc:2229] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:26.894550 19424 raft_consensus.cc:2258] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:26.895046 19424 ts_tablet_manager.cc:1905] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:26.895823 18961 tablet_service.cc:1467] Processing CreateTablet for tablet c972ce43fbaf4d56b300df1ff528a7d7 (DEFAULT_TABLE table=test-table [id=1e8392f9ad1b450291d05fa6cb259379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:26.895941 18961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c972ce43fbaf4d56b300df1ff528a7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:26.896919 19424 ts_tablet_manager.cc:1918] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:26.896981 19424 log.cc:1196] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/a3c2adc2e6994ea3a7611c8dc5c07c33
I20260812 14:42:26.897233 19424 ts_tablet_manager.cc:1939] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 227484a3577b4c87883349a15c4c6fb1: Deleting consensus metadata
I20260812 14:42:26.897456 19378 tablet_bootstrap.cc:492] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: Bootstrap starting.
I20260812 14:42:26.897616 18409 catalog_manager.cc:4841] TS 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194:37111): tablet a3c2adc2e6994ea3a7611c8dc5c07c33 (table test-table [id=16ea3d83b81344aea28e6a1f6cbdd1b1]) successfully deleted
I20260812 14:42:26.897770 18676 tablet_service.cc:1467] Processing CreateTablet for tablet c972ce43fbaf4d56b300df1ff528a7d7 (DEFAULT_TABLE table=test-table [id=1e8392f9ad1b450291d05fa6cb259379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:26.897862 18676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c972ce43fbaf4d56b300df1ff528a7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:26.898640 19378 tablet_bootstrap.cc:654] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:26.899252 19375 tablet_bootstrap.cc:492] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: Bootstrap starting.
I20260812 14:42:26.899825 19219 tablet_service.cc:1467] Processing CreateTablet for tablet c972ce43fbaf4d56b300df1ff528a7d7 (DEFAULT_TABLE table=test-table [id=1e8392f9ad1b450291d05fa6cb259379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:26.899945 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c972ce43fbaf4d56b300df1ff528a7d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:26.900107 19375 tablet_bootstrap.cc:654] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:26.900116 19378 tablet_bootstrap.cc:492] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: No bootstrap required, opened a new log
I20260812 14:42:26.900157 19378 ts_tablet_manager.cc:1397] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 14:42:26.900635 19378 raft_consensus.cc:348] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.900704 19378 raft_consensus.cc:374] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:26.900725 19378 raft_consensus.cc:729] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227484a3577b4c87883349a15c4c6fb1, State: Initialized, Role: FOLLOWER
I20260812 14:42:26.900770 19378 consensus_queue.cc:260] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.900918 19378 ts_tablet_manager.cc:1428] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:26.901558 19377 tablet_bootstrap.cc:492] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: Bootstrap starting.
I20260812 14:42:26.902160 19425 tablet_replica.cc:330] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: stopping tablet replica
I20260812 14:42:26.902313 19425 raft_consensus.cc:2229] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:26.902519 19425 raft_consensus.cc:2258] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:26.902943 19377 tablet_bootstrap.cc:654] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:26.903062 19425 ts_tablet_manager.cc:1905] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:26.904057 19377 tablet_bootstrap.cc:492] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: No bootstrap required, opened a new log
I20260812 14:42:26.904111 19377 ts_tablet_manager.cc:1397] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260812 14:42:26.904668 19377 raft_consensus.cc:348] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.904742 19377 raft_consensus.cc:374] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:26.904760 19377 raft_consensus.cc:729] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50b3ace997448ac96aeda06caeb8c33, State: Initialized, Role: FOLLOWER
I20260812 14:42:26.904852 19377 consensus_queue.cc:260] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.904985 19377 ts_tablet_manager.cc:1428] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:26.905048 19425 ts_tablet_manager.cc:1918] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:26.905107 19425 log.cc:1196] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/a3c2adc2e6994ea3a7611c8dc5c07c33
I20260812 14:42:26.905287 19425 ts_tablet_manager.cc:1939] T a3c2adc2e6994ea3a7611c8dc5c07c33 P f50b3ace997448ac96aeda06caeb8c33: Deleting consensus metadata
I20260812 14:42:26.905730 18409 catalog_manager.cc:4841] TS f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579): tablet a3c2adc2e6994ea3a7611c8dc5c07c33 (table test-table [id=16ea3d83b81344aea28e6a1f6cbdd1b1]) successfully deleted
I20260812 14:42:26.905833 19375 tablet_bootstrap.cc:492] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: No bootstrap required, opened a new log
I20260812 14:42:26.905884 19375 ts_tablet_manager.cc:1397] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260812 14:42:26.906203 19427 tablet_replica.cc:330] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: stopping tablet replica
I20260812 14:42:26.906308 19427 raft_consensus.cc:2229] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:26.906294 19375 raft_consensus.cc:348] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.906433 19375 raft_consensus.cc:374] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:26.906466 19375 raft_consensus.cc:729] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d68200041b4837b5ede77784a09e4f, State: Initialized, Role: FOLLOWER
I20260812 14:42:26.906473 19427 raft_consensus.cc:2258] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:26.906515 19375 consensus_queue.cc:260] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.906592 19375 ts_tablet_manager.cc:1428] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:26.907051 19427 ts_tablet_manager.cc:1905] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:26.908967 19427 ts_tablet_manager.cc:1918] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:26.909030 19427 log.cc:1196] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/a3c2adc2e6994ea3a7611c8dc5c07c33
I20260812 14:42:26.909229 19427 ts_tablet_manager.cc:1939] T a3c2adc2e6994ea3a7611c8dc5c07c33 P 81d68200041b4837b5ede77784a09e4f: Deleting consensus metadata
I20260812 14:42:26.909544 18409 catalog_manager.cc:4841] TS 81d68200041b4837b5ede77784a09e4f (127.15.227.193:35417): tablet a3c2adc2e6994ea3a7611c8dc5c07c33 (table test-table [id=16ea3d83b81344aea28e6a1f6cbdd1b1]) successfully deleted
I20260812 14:42:26.960819 19394 raft_consensus.cc:482] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:26.960923 19394 raft_consensus.cc:504] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.961076 19394 leader_election.cc:288] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194:37111), f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579)
I20260812 14:42:26.961297 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c972ce43fbaf4d56b300df1ff528a7d7" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227484a3577b4c87883349a15c4c6fb1" is_pre_election: true
I20260812 14:42:26.961311 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c972ce43fbaf4d56b300df1ff528a7d7" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50b3ace997448ac96aeda06caeb8c33" is_pre_election: true
I20260812 14:42:26.961393 18997 raft_consensus.cc:2454] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 0.
I20260812 14:42:26.961410 19254 raft_consensus.cc:2454] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 0.
I20260812 14:42:26.961573 18584 leader_election.cc:302] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227484a3577b4c87883349a15c4c6fb1, 81d68200041b4837b5ede77784a09e4f; no voters:
I20260812 14:42:26.961680 19394 raft_consensus.cc:2781] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:26.961737 19394 raft_consensus.cc:482] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:26.961762 19394 raft_consensus.cc:3037] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:26.963049 19394 raft_consensus.cc:504] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.963177 19394 leader_election.cc:288] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 election: Requested vote from peers 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194:37111), f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579)
I20260812 14:42:26.963339 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c972ce43fbaf4d56b300df1ff528a7d7" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50b3ace997448ac96aeda06caeb8c33"
I20260812 14:42:26.963343 18997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c972ce43fbaf4d56b300df1ff528a7d7" candidate_uuid: "81d68200041b4837b5ede77784a09e4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "227484a3577b4c87883349a15c4c6fb1"
I20260812 14:42:26.963418 18997 raft_consensus.cc:3037] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:26.963418 19254 raft_consensus.cc:3037] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:26.964134 18997 raft_consensus.cc:2454] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 1.
I20260812 14:42:26.964416 19254 raft_consensus.cc:2454] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81d68200041b4837b5ede77784a09e4f in term 1.
I20260812 14:42:26.964588 18584 leader_election.cc:302] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227484a3577b4c87883349a15c4c6fb1, 81d68200041b4837b5ede77784a09e4f; no voters:
I20260812 14:42:26.964679 19394 raft_consensus.cc:2781] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:26.964746 19394 raft_consensus.cc:686] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 1 LEADER]: Becoming Leader. State: Replica: 81d68200041b4837b5ede77784a09e4f, State: Running, Role: LEADER
I20260812 14:42:26.964799 19394 consensus_queue.cc:237] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:26.965361 18434 catalog_manager.cc:5495] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 81d68200041b4837b5ede77784a09e4f (127.15.227.193). New cstate: current_term: 1 leader_uuid: "81d68200041b4837b5ede77784a09e4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } health_report { overall_health: UNKNOWN } } }
Aug 12 14:42:26 dist-test-slave-q9h9 krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:26.997267 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.986799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46064 (local address 127.15.227.254:39443)
0812 14:42:26.986846 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:26.986848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:26.987096 (+ 248us) server_negotiation.cc:407] Connection header received
0812 14:42:26.987126 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:26.987127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:26.987138 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:26.987195 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:26.987636 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.988375 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.989099 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.989390 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.989467 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.994963 (+ 5496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:26.994969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:26.994970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:26.994977 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:26.996206 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.996522 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.996523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.996523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.996569 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:26.996749 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:26.996750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:26.996751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:26.996861 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:26.996920 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:26.997057 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 14:42:26.997139 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:42:26.997313 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:26.986759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:39443 (local address 127.0.0.1:46064)
0812 14:42:26.987020 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:26.987030 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:26.987093 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:26.987208 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:26.987209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:26.987516 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:26.987561 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.987563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.988475 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.988476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.989010 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:26.989012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:26.989462 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:26.989463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:26.989503 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:26.993397 (+ 3894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:26.993402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:26.994865 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:26.996271 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.996279 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.996282 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.996462 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.996619 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:26.996620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:26.996620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:26.996674 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:26.996914 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:26.996922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:26.996970 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:42:26.997029 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:42:26.997195 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":52,"thread_start_us":152,"threads_started":1}
I20260812 14:42:26.999500 18434 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:46064:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:42:26.999744 18434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:27.001168 18961 tablet_service.cc:1467] Processing CreateTablet for tablet c567808e27fd4510b34b917af5a77ad0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b8cf5250ab448d3b4a482a76266738f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:27.001353 18961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c567808e27fd4510b34b917af5a77ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:27.002169 18676 tablet_service.cc:1467] Processing CreateTablet for tablet c567808e27fd4510b34b917af5a77ad0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b8cf5250ab448d3b4a482a76266738f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:27.002307 18676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c567808e27fd4510b34b917af5a77ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:27.002530 19378 tablet_bootstrap.cc:492] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: Bootstrap starting.
I20260812 14:42:27.003377 19378 tablet_bootstrap.cc:654] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:27.004004 19375 tablet_bootstrap.cc:492] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: Bootstrap starting.
I20260812 14:42:27.004078 19219 tablet_service.cc:1467] Processing CreateTablet for tablet c567808e27fd4510b34b917af5a77ad0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3b8cf5250ab448d3b4a482a76266738f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:27.004215 19219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c567808e27fd4510b34b917af5a77ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:27.004247 19375 tablet_bootstrap.cc:654] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:27.004758 19375 tablet_bootstrap.cc:492] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: No bootstrap required, opened a new log
I20260812 14:42:27.004794 19375 ts_tablet_manager.cc:1397] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260812 14:42:27.004897 19375 raft_consensus.cc:348] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.004941 19375 raft_consensus.cc:374] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:27.004953 19375 raft_consensus.cc:729] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81d68200041b4837b5ede77784a09e4f, State: Initialized, Role: FOLLOWER
I20260812 14:42:27.004990 19375 consensus_queue.cc:260] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.005076 19375 ts_tablet_manager.cc:1428] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:27.005095 19394 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.005162 19394 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:27.005581 19377 tablet_bootstrap.cc:492] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: Bootstrap starting.
I20260812 14:42:27.005923 19377 tablet_bootstrap.cc:654] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:27.006522 19378 tablet_bootstrap.cc:492] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: No bootstrap required, opened a new log
I20260812 14:42:27.006600 19378 ts_tablet_manager.cc:1397] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260812 14:42:27.006615 19377 tablet_bootstrap.cc:492] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: No bootstrap required, opened a new log
I20260812 14:42:27.006654 19377 ts_tablet_manager.cc:1397] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:27.006757 19378 raft_consensus.cc:348] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.006776 19377 raft_consensus.cc:348] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.006819 19378 raft_consensus.cc:374] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:27.006827 19377 raft_consensus.cc:374] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:27.006840 19377 raft_consensus.cc:729] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f50b3ace997448ac96aeda06caeb8c33, State: Initialized, Role: FOLLOWER
I20260812 14:42:27.006839 19378 raft_consensus.cc:729] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227484a3577b4c87883349a15c4c6fb1, State: Initialized, Role: FOLLOWER
I20260812 14:42:27.006877 19377 consensus_queue.cc:260] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.006883 19378 consensus_queue.cc:260] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.006958 19377 ts_tablet_manager.cc:1428] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:27.006973 19378 ts_tablet_manager.cc:1428] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:27.007054 19389 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.007154 19389 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:27.007128 19388 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.007212 19388 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:27.040912 19389 raft_consensus.cc:482] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:27.041044 19389 raft_consensus.cc:504] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.041355 19389 leader_election.cc:288] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81d68200041b4837b5ede77784a09e4f (127.15.227.193:35417), f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579)
I20260812 14:42:27.044521 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.041560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:45047 (local address 127.15.227.193:35417)
0812 14:42:27.041621 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:27.041623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:27.041751 (+ 128us) server_negotiation.cc:407] Connection header received
0812 14:42:27.041899 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:27.041901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:27.041916 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:27.041964 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:27.042205 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.042692 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.043819 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.044182 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.044239 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.044248 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:27.044420 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 14:42:27.044456 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:42:27.044523 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.041831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:49821 (local address 127.15.227.195:35579)
0812 14:42:27.041888 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:27.041889 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:27.041945 (+ 56us) server_negotiation.cc:407] Connection header received
0812 14:42:27.041993 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:27.041994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:27.042004 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:27.042073 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:27.042268 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.042716 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.043820 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.044191 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.044256 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.044265 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:27.044417 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 14:42:27.044455 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:42:27.044548 19456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.041726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35579 (local address 127.15.227.194:49821)
0812 14:42:27.041848 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:27.041859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:27.041933 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:27.042128 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:27.042129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:27.042131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:27.042201 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.042203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.042798 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.042799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.043729 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.043732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.044284 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.044286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.044351 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.044355 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:27.044404 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:42:27.044443 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":69,"thread_start_us":82,"threads_started":1}
I20260812 14:42:27.044556 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.041504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:35417 (local address 127.15.227.194:45047)
0812 14:42:27.041672 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:27.041683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:27.041800 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:27.042054 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:27.042055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:27.042058 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:27.042134 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.042136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.042774 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.042775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.043728 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.043731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.044279 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.044280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.044353 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.044356 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:27.044403 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:42:27.044447 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":23,"thread_start_us":109,"threads_started":1}
I20260812 14:42:27.044926 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c567808e27fd4510b34b917af5a77ad0" candidate_uuid: "227484a3577b4c87883349a15c4c6fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50b3ace997448ac96aeda06caeb8c33" is_pre_election: true
I20260812 14:42:27.045023 19254 raft_consensus.cc:2454] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227484a3577b4c87883349a15c4c6fb1 in term 0.
I20260812 14:42:27.045004 18720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c567808e27fd4510b34b917af5a77ad0" candidate_uuid: "227484a3577b4c87883349a15c4c6fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d68200041b4837b5ede77784a09e4f" is_pre_election: true
I20260812 14:42:27.045141 18720 raft_consensus.cc:2454] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227484a3577b4c87883349a15c4c6fb1 in term 0.
I20260812 14:42:27.045218 18859 leader_election.cc:302] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227484a3577b4c87883349a15c4c6fb1, f50b3ace997448ac96aeda06caeb8c33; no voters:
I20260812 14:42:27.045356 19389 raft_consensus.cc:2781] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:27.045397 19389 raft_consensus.cc:482] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:27.045418 19389 raft_consensus.cc:3037] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:27.046151 19389 raft_consensus.cc:504] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.046271 19389 leader_election.cc:288] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [CANDIDATE]: Term 1 election: Requested vote from peers 81d68200041b4837b5ede77784a09e4f (127.15.227.193:35417), f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579)
I20260812 14:42:27.046451 19254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c567808e27fd4510b34b917af5a77ad0" candidate_uuid: "227484a3577b4c87883349a15c4c6fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f50b3ace997448ac96aeda06caeb8c33"
I20260812 14:42:27.046458 18720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c567808e27fd4510b34b917af5a77ad0" candidate_uuid: "227484a3577b4c87883349a15c4c6fb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81d68200041b4837b5ede77784a09e4f"
I20260812 14:42:27.046528 19254 raft_consensus.cc:3037] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:27.046530 18720 raft_consensus.cc:3037] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:27.047057 19254 raft_consensus.cc:2454] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227484a3577b4c87883349a15c4c6fb1 in term 1.
I20260812 14:42:27.047223 18859 leader_election.cc:302] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227484a3577b4c87883349a15c4c6fb1, f50b3ace997448ac96aeda06caeb8c33; no voters:
I20260812 14:42:27.047250 18720 raft_consensus.cc:2454] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227484a3577b4c87883349a15c4c6fb1 in term 1.
I20260812 14:42:27.047346 19389 raft_consensus.cc:2781] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:27.047466 19389 raft_consensus.cc:686] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [term 1 LEADER]: Becoming Leader. State: Replica: 227484a3577b4c87883349a15c4c6fb1, State: Running, Role: LEADER
I20260812 14:42:27.047533 19389 consensus_queue.cc:237] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } }
I20260812 14:42:27.047894 19457 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: TxnStatusTablet state changed. Reason: New leader 227484a3577b4c87883349a15c4c6fb1. Latest consensus state: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.047983 19457 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:27.048167 19460 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:42:27.048262 19460 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:42:27.048237 18434 catalog_manager.cc:5495] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194). New cstate: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:27.055037 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.052775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.15.227.194:37111)
0812 14:42:27.052817 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:27.052820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:27.052853 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:42:27.052930 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:27.052931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:27.052948 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:27.053001 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:27.053114 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.053568 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.054341 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.054530 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.054582 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.054746 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:27.054848 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:27.054908 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:27.054957 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 14:42:27.054976 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:42:27.055070 19443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.052724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:37111 (local address 127.0.0.1:53174)
0812 14:42:27.052813 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:27.052823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:27.052870 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:27.053002 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:27.053003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:27.053004 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:27.053053 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.053055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.053637 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.053638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.054269 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.054272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.054587 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.054589 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.054655 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.054670 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:27.054903 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:27.054912 (+ 9us) client_negotiation.cc:769] Sending connection context
0812 14:42:27.054955 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:42:27.055004 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 14:42:27.056103 19254 raft_consensus.cc:1261] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Refusing update from remote peer 227484a3577b4c87883349a15c4c6fb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:27.056238 18720 raft_consensus.cc:1261] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Refusing update from remote peer 227484a3577b4c87883349a15c4c6fb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:27.056336 19389 consensus_queue.cc:1035] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:27.056473 19457 consensus_queue.cc:1035] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:27.056926 19388 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: TxnStatusTablet state changed. Reason: New leader 227484a3577b4c87883349a15c4c6fb1. Latest consensus state: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.057014 19388 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:27.057483 19394 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: TxnStatusTablet state changed. Reason: New leader 227484a3577b4c87883349a15c4c6fb1. Latest consensus state: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.057574 19394 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:27.058087 19389 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.058167 19389 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:27.058203 19389 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.058240 19389 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P 227484a3577b4c87883349a15c4c6fb1: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:27.058534 19388 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.058614 19388 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P f50b3ace997448ac96aeda06caeb8c33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:27.059262 19394 tablet_replica.cc:439] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "227484a3577b4c87883349a15c4c6fb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "81d68200041b4837b5ede77784a09e4f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 35417 } } peers { permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 } } peers { permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 } } }
I20260812 14:42:27.059345 19394 tablet_replica.cc:442] T c567808e27fd4510b34b917af5a77ad0 P 81d68200041b4837b5ede77784a09e4f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:27.064983 19478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.062497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:37111 (local address 127.15.227.193:47053)
0812 14:42:27.062658 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:27.062668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:27.062719 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:27.062832 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:27.062832 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:27.062833 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:27.062869 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.062870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.063662 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.063663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.064264 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.064266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.064675 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.064676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.064716 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.064731 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:27.064872 (+ 141us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:27.064873 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:42:27.064899 (+ 26us) client_negotiation.cc:240] Negotiation successful
0812 14:42:27.064931 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260812 14:42:27.064985 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.062560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:47053 (local address 127.15.227.194:37111)
0812 14:42:27.062599 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:27.062601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:27.062721 (+ 120us) server_negotiation.cc:407] Connection header received
0812 14:42:27.062758 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:27.062758 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:27.062767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:27.062830 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:27.062903 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.063599 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.064326 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.064611 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.064674 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.064793 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:27.064822 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:27.064872 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:27.064905 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 14:42:27.064922 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:42:27.067026 18997 raft_consensus.cc:1261] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Refusing update from remote peer 81d68200041b4837b5ede77784a09e4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:27.067042 19254 raft_consensus.cc:1261] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Refusing update from remote peer 81d68200041b4837b5ede77784a09e4f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:27.067256 19394 consensus_queue.cc:1035] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f50b3ace997448ac96aeda06caeb8c33" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:27.067384 19421 consensus_queue.cc:1035] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "227484a3577b4c87883349a15c4c6fb1" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 37111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:27.069646 19400 mvcc.cc:205] Tried to move back new op lower bound from 7317691379985068032 to 7317691379568218112. Current Snapshot: MvccSnapshot[applied={T|T < 7317691379985068032}]
I20260812 14:42:27.076311 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.073808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:35417 (local address 127.15.227.194:49715)
0812 14:42:27.073950 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:27.073960 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:27.074031 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:27.074148 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:27.074149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:27.074150 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:27.074212 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.074213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.074796 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.074797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.075512 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:27.075514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.075980 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.075981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:27.076034 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.076047 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:27.076191 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:27.076192 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:42:27.076219 (+ 27us) client_negotiation.cc:240] Negotiation successful
0812 14:42:27.076247 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260812 14:42:27.076313 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.073853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:49715 (local address 127.15.227.193:35417)
0812 14:42:27.073888 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:27.073890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:27.074011 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:42:27.074085 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:27.074086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:27.074095 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:27.074145 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:27.074263 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.074717 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.075603 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.075903 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.075961 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.076108 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:27.076140 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:27.076191 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:27.076229 (+ 38us) server_negotiation.cc:299] Negotiation successful
0812 14:42:27.076249 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:42:27.084975 18436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:27.085054 18436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:27.085661 18436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e4031bacc2349628214f6d8f83e4b8a: Sending DeleteTablet for 3 replicas of tablet c972ce43fbaf4d56b300df1ff528a7d7
I20260812 14:42:27.085898 18676 tablet_service.cc:1514] Processing DeleteTablet for tablet c972ce43fbaf4d56b300df1ff528a7d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:27 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55672
I20260812 14:42:27.085985 19427 tablet_replica.cc:330] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: stopping tablet replica
I20260812 14:42:27.086050 19427 raft_consensus.cc:2229] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:27.086035 19219 tablet_service.cc:1514] Processing DeleteTablet for tablet c972ce43fbaf4d56b300df1ff528a7d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:27 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:43870
I20260812 14:42:27.086105 19427 raft_consensus.cc:2258] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:27.086174 19425 tablet_replica.cc:330] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: stopping tablet replica
I20260812 14:42:27.086225 19425 raft_consensus.cc:2229] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:27.086256 19425 raft_consensus.cc:2258] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:27.086663 19427 ts_tablet_manager.cc:1905] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:27.086663 19425 ts_tablet_manager.cc:1905] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:27.088167 18961 tablet_service.cc:1514] Processing DeleteTablet for tablet c972ce43fbaf4d56b300df1ff528a7d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:27 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:53158
I20260812 14:42:27.088258 19424 tablet_replica.cc:330] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: stopping tablet replica
I20260812 14:42:27.088304 19424 raft_consensus.cc:2229] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:27.088342 19424 raft_consensus.cc:2258] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:27.088764 19424 ts_tablet_manager.cc:1905] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:27.088795 19427 ts_tablet_manager.cc:1918] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:27.088848 19427 log.cc:1196] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/c972ce43fbaf4d56b300df1ff528a7d7
I20260812 14:42:27.089079 19427 ts_tablet_manager.cc:1939] T c972ce43fbaf4d56b300df1ff528a7d7 P 81d68200041b4837b5ede77784a09e4f: Deleting consensus metadata
I20260812 14:42:27.089365 19425 ts_tablet_manager.cc:1918] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:27.089421 19425 log.cc:1196] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/c972ce43fbaf4d56b300df1ff528a7d7
I20260812 14:42:27.089397 18409 catalog_manager.cc:4841] TS 81d68200041b4837b5ede77784a09e4f (127.15.227.193:35417): tablet c972ce43fbaf4d56b300df1ff528a7d7 (table test-table [id=1e8392f9ad1b450291d05fa6cb259379]) successfully deleted
I20260812 14:42:27.089619 19425 ts_tablet_manager.cc:1939] T c972ce43fbaf4d56b300df1ff528a7d7 P f50b3ace997448ac96aeda06caeb8c33: Deleting consensus metadata
I20260812 14:42:27.089913 18409 catalog_manager.cc:4841] TS f50b3ace997448ac96aeda06caeb8c33 (127.15.227.195:35579): tablet c972ce43fbaf4d56b300df1ff528a7d7 (table test-table [id=1e8392f9ad1b450291d05fa6cb259379]) successfully deleted
I20260812 14:42:27.090153 19424 ts_tablet_manager.cc:1918] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:27.090200 19424 log.cc:1196] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/c972ce43fbaf4d56b300df1ff528a7d7
I20260812 14:42:27.090371 19424 ts_tablet_manager.cc:1939] T c972ce43fbaf4d56b300df1ff528a7d7 P 227484a3577b4c87883349a15c4c6fb1: Deleting consensus metadata
I20260812 14:42:27.090754 18409 catalog_manager.cc:4841] TS 227484a3577b4c87883349a15c4c6fb1 (127.15.227.194:37111): tablet c972ce43fbaf4d56b300df1ff528a7d7 (table test-table [id=1e8392f9ad1b450291d05fa6cb259379]) successfully deleted
Aug 12 14:42:27 dist-test-slave-q9h9 krb5kdc[18351](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545746, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:27.096305 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.086612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.15.227.193:35417)
0812 14:42:27.086649 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:27.086650 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:27.086902 (+ 252us) server_negotiation.cc:407] Connection header received
0812 14:42:27.086990 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:27.086991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:27.087002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:27.087029 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:27.087803 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.088540 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.090962 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.091144 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.091198 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.093743 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:27.093752 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:27.093752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:27.093759 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:27.095255 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:27.095611 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:27.095613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:27.095614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:27.095666 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:27.095848 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:27.095849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:27.095849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:27.095975 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:27.096097 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:27.096172 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 14:42:27.096241 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:42:27.096580 18785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55686
I20260812 14:42:27.102689 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:27.097435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46074 (local address 127.15.227.254:39443)
0812 14:42:27.097475 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:27.097477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:27.097543 (+ 66us) server_negotiation.cc:407] Connection header received
0812 14:42:27.097627 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:27.097628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:27.097638 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:27.097664 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:27.098181 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.098652 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.099288 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:27.099486 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:27.099539 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:27.100506 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:27.100507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:27.100507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:27.100514 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:27.101705 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:27.101980 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:27.101981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:27.101982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:27.102083 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:27.102252 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:27.102253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:27.102254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:27.102357 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:27.102433 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:27.102533 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:42:27.102610 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:42:27.102876 18436 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46074
I20260812 14:42:27.103404 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18555
I20260812 14:42:27.124565 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18824
I20260812 14:42:27.138357 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 19096
I20260812 14:42:27.150614 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18372
2026-08-12T14:42:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2460 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:27 dist-test-slave-q9h9 krb5kdc[19501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:27 dist-test-slave-q9h9 krb5kdc[19501](info): set up 2 sockets
Aug 12 14:42:27 dist-test-slave-q9h9 krb5kdc[19501](info): commencing operation
krb5kdc: starting...
W20260812 14:42:28.351783 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:28.461230 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37481
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:28.540809 19527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:28.540992 19527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:28.541013 19527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:28.542135 19527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:28.542172 19527 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:42:28.542184 19527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:28.542207 19527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:28.542232 19527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:28.542246 19527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:28.543676 19527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:37481
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:28.543893 19527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:28.544094 19527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:28.548612 19548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:28.548693 19546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:28.548628 19545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:28.548861 19527 server_base.cc:1034] running on GCE node
I20260812 14:42:28.549126 19527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:28.549381 19527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:28.550532 19527 hybrid_clock.cc:648] HybridClock initialized: now 1786545748550518 us; error 28 us; skew 500 ppm
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:28.552953 19527 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:28.553345 19527 webserver.cc:460] Webserver started at http://127.15.227.254:46377/ using document root <none> and password file <none>
I20260812 14:42:28.553510 19527 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:28.553547 19527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:28.553634 19527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:28.554621 19527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "b380fb00152b4f968725ebbcd21c0546"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "f6ea5c345689b186e54718716da4eebe"
server_key_iv: "da97efda6342e3ad43102047ac40a199"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.554926 19527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "b380fb00152b4f968725ebbcd21c0546"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "f6ea5c345689b186e54718716da4eebe"
server_key_iv: "da97efda6342e3ad43102047ac40a199"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.556113 19527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:28.556770 19560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:28.556918 19527 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:28.557001 19527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "b380fb00152b4f968725ebbcd21c0546"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "f6ea5c345689b186e54718716da4eebe"
server_key_iv: "da97efda6342e3ad43102047ac40a199"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.557058 19527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:28.568943 19527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:28.570686 19527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:28.570808 19527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:28.575227 19527 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:37481
I20260812 14:42:28.575313 19684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:37481 every 8 connection(s)
I20260812 14:42:28.575565 19527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:28.576162 19687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:28.578286 19687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Bootstrap starting.
I20260812 14:42:28.578851 19687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:28.579075 19687 log.cc:824] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:28.579661 19687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: No bootstrap required, opened a new log
I20260812 14:42:28.581780 19687 raft_consensus.cc:348] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b380fb00152b4f968725ebbcd21c0546" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37481 } }
I20260812 14:42:28.581908 19687 raft_consensus.cc:374] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:28.581929 19687 raft_consensus.cc:729] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b380fb00152b4f968725ebbcd21c0546, State: Initialized, Role: FOLLOWER
I20260812 14:42:28.582055 19687 consensus_queue.cc:260] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b380fb00152b4f968725ebbcd21c0546" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37481 } }
I20260812 14:42:28.582130 19687 raft_consensus.cc:388] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:28.582159 19687 raft_consensus.cc:482] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:28.582192 19687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:28.582646 19687 raft_consensus.cc:504] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b380fb00152b4f968725ebbcd21c0546" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37481 } }
I20260812 14:42:28.582727 19687 leader_election.cc:302] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b380fb00152b4f968725ebbcd21c0546; no voters:
I20260812 14:42:28.582862 19687 leader_election.cc:288] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:28.582916 19691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:28.583041 19691 raft_consensus.cc:686] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [term 1 LEADER]: Becoming Leader. State: Replica: b380fb00152b4f968725ebbcd21c0546, State: Running, Role: LEADER
I20260812 14:42:28.583108 19687 sys_catalog.cc:564] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:28.583235 19691 consensus_queue.cc:237] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b380fb00152b4f968725ebbcd21c0546" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37481 } }
I20260812 14:42:28.583597 19692 sys_catalog.cc:455] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b380fb00152b4f968725ebbcd21c0546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b380fb00152b4f968725ebbcd21c0546" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37481 } } }
I20260812 14:42:28.583694 19692 sys_catalog.cc:458] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:28.583783 19696 sys_catalog.cc:455] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b380fb00152b4f968725ebbcd21c0546. Latest consensus state: current_term: 1 leader_uuid: "b380fb00152b4f968725ebbcd21c0546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b380fb00152b4f968725ebbcd21c0546" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37481 } } }
I20260812 14:42:28.583992 19696 sys_catalog.cc:458] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:28.584162 19708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:28.584633 19708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:28.585099 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 19527
I20260812 14:42:28.585194 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:28.585361 16271 external_mini_cluster.cc:1428] Setting key dcc0761e7ca39baccf6d325b478ec494
I20260812 14:42:28.586107 19708 catalog_manager.cc:1347] Generated new cluster ID: d4a4484b6c41413fa4600ccea0dd0772
I20260812 14:42:28.586160 19708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:28.590843 19708 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:28.591411 19708 catalog_manager.cc:1504] Loading token signing keys...
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:28.599499 19708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Generated new TSK 0
I20260812 14:42:28.599660 19708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:42:28.600397 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.585947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48506 (local address 127.15.227.254:37481)
0812 14:42:28.586177 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:28.586186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:28.586219 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:42:28.590271 (+ 4052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:28.590279 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:28.590335 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:28.590481 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:28.591223 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.592641 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.593645 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.594257 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.594352 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.596912 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:28.596916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:28.596916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:28.596927 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:28.598713 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.599090 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.599093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.599093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.599196 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.599505 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.599506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.599506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.599797 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:28.599922 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:28.600024 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 14:42:28.600168 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:28.624521 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37481
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:28.704797 19731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:28.704994 19731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:28.705029 19731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:28.706197 19731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:28.706238 19731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:28.706266 19731 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:42:28.706282 19731 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:28.706310 19731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:28.707783 19731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:28.708016 19731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:28.708237 19731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:28.713039 19738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:28.713100 19731 server_base.cc:1034] running on GCE node
W20260812 14:42:28.713047 19741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:28.713022 19739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:28.713399 19731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:28.713615 19731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:28.714782 19731 hybrid_clock.cc:648] HybridClock initialized: now 1786545748714765 us; error 28 us; skew 500 ppm
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:28.717131 19731 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:28.717538 19731 webserver.cc:460] Webserver started at http://127.15.227.193:38713/ using document root <none> and password file <none>
I20260812 14:42:28.717754 19731 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:28.717803 19731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:28.717926 19731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:28.718925 19731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "8d1afc83f9a84a94b8bed6a748239727"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "1fc9b1a1002c9d8fd3db9e07e99df47a"
server_key_iv: "e6f1be45ac24d2cecb1c17fac9c1f6f4"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.719235 19731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "8d1afc83f9a84a94b8bed6a748239727"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "1fc9b1a1002c9d8fd3db9e07e99df47a"
server_key_iv: "e6f1be45ac24d2cecb1c17fac9c1f6f4"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.720420 19731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:28.721064 19751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:28.721192 19731 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:28.721274 19731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "8d1afc83f9a84a94b8bed6a748239727"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "1fc9b1a1002c9d8fd3db9e07e99df47a"
server_key_iv: "e6f1be45ac24d2cecb1c17fac9c1f6f4"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.721333 19731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:28.731412 19731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:28.732939 19731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:28.733062 19731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:28.733546 19731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:28.733592 19731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:28.733618 19731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:28.733644 19731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:28.740613 19731 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:33175
I20260812 14:42:28.740680 19998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:33175 every 8 connection(s)
I20260812 14:42:28.740993 19731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:28.742837 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.733885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:49995 (local address 127.15.227.254:37481)
0812 14:42:28.733951 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:28.733953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:28.734180 (+ 227us) server_negotiation.cc:407] Connection header received
0812 14:42:28.734771 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:28.734773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:28.734782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:28.734828 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:28.735618 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.736141 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.737608 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.737818 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.737867 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.740374 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:28.740376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:28.740376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:28.740383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:28.741738 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.742153 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.742154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.742154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.742207 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.742392 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.742392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.742393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.742485 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:28.742529 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:28.742709 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 14:42:28.742775 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:42:28.742980 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.733864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37481 (local address 127.15.227.193:49995)
0812 14:42:28.734037 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:28.734091 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:28.734672 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:28.734917 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:28.734921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:28.735266 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:28.735538 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.735544 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.736235 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.736236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.737502 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.737505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.737890 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.737891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.738083 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.738555 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:28.738564 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:28.740288 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:28.741813 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.741816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.741817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.742094 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.742264 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.742264 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.742264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.742355 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.742523 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:28.742526 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:28.742623 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 14:42:28.742690 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:42:28.742800 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260812 14:42:28.743434 19999 heartbeater.cc:347] Connected to a master server at 127.15.227.254:37481
I20260812 14:42:28.743512 19999 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:28.743665 19999 heartbeater.cc:511] Master 127.15.227.254:37481 requested a full tablet report, sending...
I20260812 14:42:28.744194 19584 ts_manager.cc:194] Registered new tserver with Master: 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175)
I20260812 14:42:28.745026 19584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:49995
I20260812 14:42:28.749010 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.744477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:49323 (local address 127.15.227.254:37481)
0812 14:42:28.744514 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:28.744516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:28.744579 (+ 63us) server_negotiation.cc:407] Connection header received
0812 14:42:28.744638 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:28.744638 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:28.744645 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:28.744685 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:28.744991 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.745620 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.746368 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.746548 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.746608 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.747302 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:28.747304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:28.747304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:28.747310 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:28.748303 (+ 993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.748560 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.748561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.748561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.748602 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.748705 (+ 103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.748705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.748705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.748782 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:28.748821 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:28.748907 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 14:42:28.748957 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:42:28.749047 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.744402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37481 (local address 127.15.227.193:49323)
0812 14:42:28.744525 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:28.744535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:28.744590 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:28.744690 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:28.744690 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:28.744876 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:28.744937 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.744938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.745717 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.745717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.746286 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.746288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.746607 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.746607 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.746654 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.746982 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:28.746985 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:28.747236 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:28.748364 (+ 1128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.748364 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.748365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.748510 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.748635 (+ 125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.748636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.748636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.748675 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.748813 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:28.748813 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:28.748843 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 14:42:28.748892 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:42:28.748956 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260812 14:42:28.749150 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 19731
I20260812 14:42:28.749224 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:28.749388 16271 external_mini_cluster.cc:1428] Setting key 35e39b8b2a06b7a5f9f1b42dc3b7de50
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:28.770833 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37481
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:28.849838 20016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:28.850047 20016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:28.850085 20016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:28.851184 20016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:28.851225 20016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:28.851259 20016 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:42:28.851276 20016 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:28.851306 20016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:28.852721 20016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:28.852954 20016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:28.853150 20016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:28.857792 20030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:28.857921 20016 server_base.cc:1034] running on GCE node
W20260812 14:42:28.857980 20038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:28.858084 20033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:28.858286 20016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:28.858508 20016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:28.859656 20016 hybrid_clock.cc:648] HybridClock initialized: now 1786545748859645 us; error 29 us; skew 500 ppm
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:28.861974 20016 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:28.862423 20016 webserver.cc:460] Webserver started at http://127.15.227.194:37391/ using document root <none> and password file <none>
I20260812 14:42:28.862632 20016 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:28.862679 20016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:28.862798 20016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:28.863756 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "c0eada9623c444afbd46294cab377db8"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "f04ca0680c252f8297c5a4a935bf9997"
server_key_iv: "4bcc475af6643b45085a755b1fb7ae25"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.864074 20016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "c0eada9623c444afbd46294cab377db8"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "f04ca0680c252f8297c5a4a935bf9997"
server_key_iv: "4bcc475af6643b45085a755b1fb7ae25"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.865326 20016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 14:42:28.866034 20047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:28.866211 20016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:28.866302 20016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "c0eada9623c444afbd46294cab377db8"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "f04ca0680c252f8297c5a4a935bf9997"
server_key_iv: "4bcc475af6643b45085a755b1fb7ae25"
server_key_version: "encryptionkey@0"
I20260812 14:42:28.866367 20016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:28.875063 20016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:28.876705 20016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:28.876832 20016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:28.877301 20016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:28.877341 20016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:28.877367 20016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:28.877403 20016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:28.884104 20016 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:34855
I20260812 14:42:28.884284 20275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:34855 every 8 connection(s)
I20260812 14:42:28.884483 20016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:28.884912 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 20016
I20260812 14:42:28.885010 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:28.885185 16271 external_mini_cluster.cc:1428] Setting key da668a42260f05a8bdef8e831f95b3bd
I20260812 14:42:28.886770 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.877686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:50349 (local address 127.15.227.254:37481)
0812 14:42:28.877763 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:28.877766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:28.877913 (+ 147us) server_negotiation.cc:407] Connection header received
0812 14:42:28.878392 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:28.878394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:28.878407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:28.878480 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:28.879103 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.879624 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.880588 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.880815 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.880871 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.883692 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:28.883694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:28.883694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:28.883701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:28.885371 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.885844 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.885845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.885846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.885905 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.886158 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.886159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.886159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.886273 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:28.886329 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:28.886591 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 14:42:28.886670 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:42:28.886878 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.877639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37481 (local address 127.15.227.194:50349)
0812 14:42:28.877793 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:28.877832 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:28.878299 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:28.878531 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:28.878534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:28.878763 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:28.878981 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.878988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.879712 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.879713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.880510 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.880512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.880885 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.880886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.881000 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.881471 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:28.881480 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:28.883616 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:28.885513 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.885518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.885520 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.885777 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.885958 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.885959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.885959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.886100 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.886330 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:28.886338 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:28.886464 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 14:42:28.886541 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:42:28.886682 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
I20260812 14:42:28.887326 20276 heartbeater.cc:347] Connected to a master server at 127.15.227.254:37481
I20260812 14:42:28.887404 20276 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:28.887564 20276 heartbeater.cc:511] Master 127.15.227.254:37481 requested a full tablet report, sending...
I20260812 14:42:28.888243 19586 ts_manager.cc:194] Registered new tserver with Master: c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855)
I20260812 14:42:28.888613 19586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:50349
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
I20260812 14:42:28.898696 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.888288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37481 (local address 127.15.227.194:38757)
0812 14:42:28.893602 (+ 5314us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:28.893612 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:28.893676 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:28.893805 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:28.893805 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:28.893958 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:28.894005 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.894006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.894643 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.894644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.895170 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:28.895172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.895497 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.895497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:28.895556 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.896009 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:28.896015 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:28.896346 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:28.897752 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.897752 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.897753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.897913 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.898119 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:28.898119 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:28.898119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:28.898174 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:28.898466 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:28.898467 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:28.898502 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 14:42:28.898546 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:28.898618 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5288,"thread_start_us":63,"threads_started":1}
I20260812 14:42:28.898721 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:28.888349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:38757 (local address 127.15.227.254:37481)
0812 14:42:28.888393 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:28.888395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:28.893675 (+ 5280us) server_negotiation.cc:407] Connection header received
0812 14:42:28.893719 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:28.893721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:28.893732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:28.893800 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:28.894077 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.894565 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.895231 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:28.895437 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:28.895496 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:28.896431 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:28.896433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:28.896433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:28.896439 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:28.897673 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.897975 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.897976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.897977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.898061 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:28.898225 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:28.898225 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:28.898226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:28.898408 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:28.898479 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:28.898573 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 14:42:28.898642 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:28.908277 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37481
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:28.987051 20296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:28.987236 20296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:28.987265 20296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:28.988397 20296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:28.988436 20296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:28.988456 20296 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 14:42:28.988466 20296 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:28.988481 20296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:28.989871 20296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:28.990111 20296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:28.990317 20296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:28.994939 20314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:28.994990 20311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:28.995049 20312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:28.995167 20296 server_base.cc:1034] running on GCE node
I20260812 14:42:28.995373 20296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:28.995600 20296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:28.996735 20296 hybrid_clock.cc:648] HybridClock initialized: now 1786545748996719 us; error 29 us; skew 500 ppm
Aug 12 14:42:28 dist-test-slave-q9h9 krb5kdc[19501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:28.999090 20296 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:28.999502 20296 webserver.cc:460] Webserver started at http://127.15.227.195:42121/ using document root <none> and password file <none>
I20260812 14:42:28.999704 20296 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:28.999748 20296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:28.999873 20296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:29.000818 20296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "aa10776f19e643b889f94d6ab9107a97"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "307f80ee0c1c29d1c77373a5e3e49d61"
server_key_iv: "0803fe7306783fb361fb4617de892782"
server_key_version: "encryptionkey@0"
I20260812 14:42:29.001137 20296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "aa10776f19e643b889f94d6ab9107a97"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "307f80ee0c1c29d1c77373a5e3e49d61"
server_key_iv: "0803fe7306783fb361fb4617de892782"
server_key_version: "encryptionkey@0"
I20260812 14:42:29.002370 20296 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:42:29.003064 20327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.003247 20296 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:29.003330 20296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "aa10776f19e643b889f94d6ab9107a97"
format_stamp: "Formatted at 2026-08-12 14:42:28 on dist-test-slave-q9h9"
server_key: "307f80ee0c1c29d1c77373a5e3e49d61"
server_key_iv: "0803fe7306783fb361fb4617de892782"
server_key_version: "encryptionkey@0"
I20260812 14:42:29.003382 20296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:29.021775 20296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:29.023813 20296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:29.023928 20296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:29.024470 20296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:29.024518 20296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.024539 20296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:29.024550 20296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.031024 20296 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:34235
Aug 12 14:42:29 dist-test-slave-q9h9 krb5kdc[19501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:29.031109 20546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:34235 every 8 connection(s)
I20260812 14:42:29.031418 20296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:42:29.032537 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 20296
I20260812 14:42:29.032634 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:29.032811 16271 external_mini_cluster.cc:1428] Setting key 1a55aac4263603fbed59598fc9ceb74b
I20260812 14:42:29.034209 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.024939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:60845 (local address 127.15.227.254:37481)
0812 14:42:29.025011 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.025014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.025209 (+ 195us) server_negotiation.cc:407] Connection header received
0812 14:42:29.025727 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.025728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.025743 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.025817 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:29.026665 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.027481 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.028969 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.029254 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.029328 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.031533 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:29.031535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:29.031535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:29.031543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:29.032936 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.033333 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.033334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.033334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.033392 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.033600 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.033601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.033601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.033710 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:29.033773 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.034038 (+ 265us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.034121 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:42:29.034382 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.024893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37481 (local address 127.15.227.195:60845)
0812 14:42:29.025074 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.025136 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.025623 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.025893 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.025897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.026258 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:29.026555 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.026563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.027585 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.027586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.028873 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.028876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.029340 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.029341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.029447 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.029919 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:29.029929 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:29.031443 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:29.033001 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:29.033008 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:29.033010 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:29.033270 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:29.033441 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:29.033442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:29.033442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:29.033547 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:29.033764 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:29.033770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:29.033901 (+ 131us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.033989 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.034161 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260812 14:42:29.034812 20548 heartbeater.cc:347] Connected to a master server at 127.15.227.254:37481
I20260812 14:42:29.034894 20548 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:29.035063 20548 heartbeater.cc:511] Master 127.15.227.254:37481 requested a full tablet report, sending...
I20260812 14:42:29.035385 19584 ts_manager.cc:194] Registered new tserver with Master: aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235)
I20260812 14:42:29.035742 19584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:60845
I20260812 14:42:29.040575 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.035822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:54247 (local address 127.15.227.254:37481)
0812 14:42:29.035860 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.035862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.036475 (+ 613us) server_negotiation.cc:407] Connection header received
0812 14:42:29.036495 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.036495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.036505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.036553 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:29.036814 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.037257 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.037849 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.038011 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.038090 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.038759 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:29.038760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:29.038760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:29.038765 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:29.039885 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.040115 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.040115 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.040115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.040155 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.040271 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.040272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.040272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.040351 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:29.040390 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.040475 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.040522 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:29.040587 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.035749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37481 (local address 127.15.227.195:54247)
0812 14:42:29.036055 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.036065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.036120 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.036561 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.036561 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.036718 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:29.036764 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.036766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.037320 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.037320 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.037800 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.037801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.038087 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.038088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.038142 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.038436 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:29.038440 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:29.038680 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:29.039940 (+ 1260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:29.039940 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:29.039940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:29.040070 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:29.040193 (+ 123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:29.040194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:29.040194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:29.040235 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:29.040387 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:29.040387 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:29.040418 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.040460 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.040528 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":60,"threads_started":1}
I20260812 14:42:29.044256 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:42:29 dist-test-slave-q9h9 krb5kdc[19501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:42:29 dist-test-slave-q9h9 krb5kdc[19501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:29.065797 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.059614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48510 (local address 127.15.227.254:37481)
0812 14:42:29.059655 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.059656 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.059714 (+ 58us) server_negotiation.cc:407] Connection header received
0812 14:42:29.059779 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.059780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.059790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.059834 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:29.060330 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.060782 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.061457 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.061631 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.061681 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.064021 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:29.064022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:29.064023 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:29.064029 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:29.065021 (+ 992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.065288 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.065289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.065289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.065331 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.065448 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.065449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.065449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.065530 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:29.065597 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.065677 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.065737 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:42:29.067142 19584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:29.068753 19584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:29.074594 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.070674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34855 (local address 127.0.0.1:52060)
0812 14:42:29.070855 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.070888 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.070951 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.071126 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.071129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.071131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:29.071192 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.071194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.072459 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.072460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.073480 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.073483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.074319 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.074320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.074395 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.074408 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.074475 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.074522 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":21,"thread_start_us":86,"threads_started":1}
I20260812 14:42:29.074625 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.070783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52060 (local address 127.15.227.194:34855)
0812 14:42:29.070960 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.070965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.070975 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:29.071036 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.071038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.071053 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.071119 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:29.071253 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.072379 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.073566 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.074234 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.074301 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.074357 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.074492 (+ 135us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.074529 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260812 14:42:29.074913 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.070689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33175 (local address 127.0.0.1:34772)
0812 14:42:29.071017 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.071027 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.071078 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.071373 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.071374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.071376 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:29.071418 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.071419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.072605 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.072606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.073680 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.073682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.074675 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.074676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.074755 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.074759 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.074814 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.074851 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260812 14:42:29.075050 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.070898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:34235 (local address 127.0.0.1:55190)
0812 14:42:29.071719 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.071729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.071778 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.071923 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.071924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.071925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:29.071963 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.071964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.072947 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.072948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.073969 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.073972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.074830 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.074831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.074893 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.074897 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.074943 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.074983 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":65,"threads_started":1}
I20260812 14:42:29.075212 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.071054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.15.227.193:33175)
0812 14:42:29.071217 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.071222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.071232 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:29.071271 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.071273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.071288 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.071370 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:29.071505 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.072538 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.073782 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.074530 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.074589 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.074640 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.075061 (+ 421us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.075106 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
I20260812 14:42:29.075270 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.071296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.15.227.195:34235)
0812 14:42:29.071481 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.071484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.071779 (+ 295us) server_negotiation.cc:407] Connection header received
0812 14:42:29.071840 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.071843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.071858 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.071920 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:29.072014 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.072864 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.074071 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.074748 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.074829 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.074877 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.075148 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.075185 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":81,"threads_started":1}
I20260812 14:42:29.075599 20145 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd9bf08fa4345b9a564432bf49fd89e (DEFAULT_TABLE table=test-table [id=2584a55457eb4a26877b46e299fb60c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:29.075809 19862 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd9bf08fa4345b9a564432bf49fd89e (DEFAULT_TABLE table=test-table [id=2584a55457eb4a26877b46e299fb60c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:29.075995 20145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd9bf08fa4345b9a564432bf49fd89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.076117 19862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd9bf08fa4345b9a564432bf49fd89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.078634 20416 tablet_service.cc:1467] Processing CreateTablet for tablet 5fd9bf08fa4345b9a564432bf49fd89e (DEFAULT_TABLE table=test-table [id=2584a55457eb4a26877b46e299fb60c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:29.078917 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fd9bf08fa4345b9a564432bf49fd89e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.079568 20593 tablet_bootstrap.cc:492] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Bootstrap starting.
I20260812 14:42:29.079921 20597 tablet_bootstrap.cc:492] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Bootstrap starting.
I20260812 14:42:29.080276 20593 tablet_bootstrap.cc:654] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.080497 20597 tablet_bootstrap.cc:654] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.080646 20593 log.cc:824] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:29.080798 20597 log.cc:824] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:29.081362 20593 tablet_bootstrap.cc:492] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: No bootstrap required, opened a new log
I20260812 14:42:29.081470 20593 ts_tablet_manager.cc:1397] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:42:29.081593 20597 tablet_bootstrap.cc:492] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: No bootstrap required, opened a new log
I20260812 14:42:29.081656 20597 ts_tablet_manager.cc:1397] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:29.081959 20601 tablet_bootstrap.cc:492] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Bootstrap starting.
I20260812 14:42:29.082844 20601 tablet_bootstrap.cc:654] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.083096 20601 log.cc:824] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:29.083531 20593 raft_consensus.cc:348] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.083701 20601 tablet_bootstrap.cc:492] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: No bootstrap required, opened a new log
I20260812 14:42:29.083787 20601 ts_tablet_manager.cc:1397] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:42:29.083787 20593 raft_consensus.cc:374] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.083663 20597 raft_consensus.cc:348] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.083838 20593 raft_consensus.cc:729] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0eada9623c444afbd46294cab377db8, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.083913 20597 raft_consensus.cc:374] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.083963 20597 raft_consensus.cc:729] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.083971 20593 consensus_queue.cc:260] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.084262 20593 ts_tablet_manager.cc:1428] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 14:42:29.084242 20597 consensus_queue.cc:260] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.084297 20276 heartbeater.cc:503] Master 127.15.227.254:37481 was elected leader, sending a full tablet report...
I20260812 14:42:29.084543 20597 ts_tablet_manager.cc:1428] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 14:42:29.084609 19999 heartbeater.cc:503] Master 127.15.227.254:37481 was elected leader, sending a full tablet report...
I20260812 14:42:29.085595 20601 raft_consensus.cc:348] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.085772 20601 raft_consensus.cc:374] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.085813 20601 raft_consensus.cc:729] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa10776f19e643b889f94d6ab9107a97, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.085918 20601 consensus_queue.cc:260] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.086151 20601 ts_tablet_manager.cc:1428] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:42:29.086218 20548 heartbeater.cc:503] Master 127.15.227.254:37481 was elected leader, sending a full tablet report...
W20260812 14:42:29.135955 20278 tablet.cc:2369] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:29.138132 20609 raft_consensus.cc:482] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:29.138230 20609 raft_consensus.cc:504] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.138500 20609 leader_election.cc:288] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855), aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235)
I20260812 14:42:29.141099 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.138737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:50233 (local address 127.15.227.194:34855)
0812 14:42:29.138802 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.138805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.138813 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:29.138840 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.138841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.138850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.138906 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:29.139070 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.139549 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.140431 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.140790 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.140851 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.140860 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.141000 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.141038 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:42:29.141100 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.138663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34855 (local address 127.15.227.193:50233)
0812 14:42:29.138715 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.138726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.138779 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.138933 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.138934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.138936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:29.138995 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.138997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.139622 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.139623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.140349 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.140351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.140860 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.140862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.140941 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.140944 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.140988 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.141027 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":2}
I20260812 14:42:29.141222 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.138840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59367 (local address 127.15.227.195:34235)
0812 14:42:29.138889 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.138891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.139001 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:42:29.139045 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.139046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.139054 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.139101 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:29.139262 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.139717 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.140581 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.140926 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.140984 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.140992 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.141126 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.141161 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:42:29.141250 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.138764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:34235 (local address 127.15.227.193:59367)
0812 14:42:29.138926 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.138936 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.138992 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.139110 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.139111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.139113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:29.139200 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.139201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.139788 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.139789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.140494 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.140496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.140998 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.140999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.141066 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.141069 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.141112 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.141158 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":1,"thread_start_us":90,"threads_started":1}
I20260812 14:42:29.141479 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd9bf08fa4345b9a564432bf49fd89e" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8" is_pre_election: true
I20260812 14:42:29.141482 20462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd9bf08fa4345b9a564432bf49fd89e" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa10776f19e643b889f94d6ab9107a97" is_pre_election: true
I20260812 14:42:29.141609 20462 raft_consensus.cc:2454] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 0.
I20260812 14:42:29.141607 20188 raft_consensus.cc:2454] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 0.
I20260812 14:42:29.141811 19755 leader_election.cc:302] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, aa10776f19e643b889f94d6ab9107a97; no voters:
I20260812 14:42:29.141929 20609 raft_consensus.cc:2781] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:29.141988 20609 raft_consensus.cc:482] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:29.142035 20609 raft_consensus.cc:3037] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.142714 20609 raft_consensus.cc:504] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.142841 20609 leader_election.cc:288] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 election: Requested vote from peers c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855), aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235)
I20260812 14:42:29.142957 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd9bf08fa4345b9a564432bf49fd89e" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8"
I20260812 14:42:29.142992 20462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fd9bf08fa4345b9a564432bf49fd89e" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa10776f19e643b889f94d6ab9107a97"
I20260812 14:42:29.143030 20188 raft_consensus.cc:3037] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.143049 20462 raft_consensus.cc:3037] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.143700 20462 raft_consensus.cc:2454] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 1.
I20260812 14:42:29.143769 20188 raft_consensus.cc:2454] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 1.
I20260812 14:42:29.143842 19755 leader_election.cc:302] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, aa10776f19e643b889f94d6ab9107a97; no voters:
I20260812 14:42:29.143924 20609 raft_consensus.cc:2781] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:29.144061 20609 raft_consensus.cc:686] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Running, Role: LEADER
I20260812 14:42:29.144143 20609 consensus_queue.cc:237] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } }
I20260812 14:42:29.144874 19586 catalog_manager.cc:5495] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193). New cstate: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } health_report { overall_health: HEALTHY } } }
Aug 12 14:42:29 dist-test-slave-q9h9 krb5kdc[19501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545748, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:29.172680 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.166598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37481 (local address 127.0.0.1:48520)
0812 14:42:29.166741 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.166750 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.166813 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.166917 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.166918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.167207 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:29.167253 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.167256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.167825 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.167826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.168353 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.168354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.168668 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.168668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.168709 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.169219 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:29.169223 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:29.170552 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:29.171753 (+ 1201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:29.171761 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:29.171763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:29.171944 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:29.172105 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:29.172106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:29.172106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:29.172156 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:29.172352 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:29.172360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:29.172404 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.172461 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.172574 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260812 14:42:29.172732 19727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.166665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48520 (local address 127.15.227.254:37481)
0812 14:42:29.166734 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.166736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.166803 (+ 67us) server_negotiation.cc:407] Connection header received
0812 14:42:29.166855 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.166856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.166864 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.166903 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:29.167308 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.167757 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.168443 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.168617 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.168658 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.170624 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:29.170625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:29.170626 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:29.170632 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:29.171686 (+ 1054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.172004 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.172005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.172006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.172054 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:29.172202 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:29.172203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:29.172203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:29.172301 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:29.172356 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.172488 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.172633 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":45}
I20260812 14:42:29.174502 19586 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:48520:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:42:29.174695 19586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:29.176059 20416 tablet_service.cc:1467] Processing CreateTablet for tablet bf9a5c2fe9ba431581eacb8fe95147fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=733c0dfb5c044b0a890450e102dedd4f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:29.176208 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf9a5c2fe9ba431581eacb8fe95147fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.176154 19862 tablet_service.cc:1467] Processing CreateTablet for tablet bf9a5c2fe9ba431581eacb8fe95147fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=733c0dfb5c044b0a890450e102dedd4f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:29.176306 19862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf9a5c2fe9ba431581eacb8fe95147fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.177292 20601 tablet_bootstrap.cc:492] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: Bootstrap starting.
I20260812 14:42:29.177484 20145 tablet_service.cc:1467] Processing CreateTablet for tablet bf9a5c2fe9ba431581eacb8fe95147fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=733c0dfb5c044b0a890450e102dedd4f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:29.177627 20145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf9a5c2fe9ba431581eacb8fe95147fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.177804 20601 tablet_bootstrap.cc:654] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.178512 20601 tablet_bootstrap.cc:492] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: No bootstrap required, opened a new log
I20260812 14:42:29.178576 20601 ts_tablet_manager.cc:1397] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:29.178721 20601 raft_consensus.cc:348] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.178802 20601 raft_consensus.cc:374] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.178825 20601 raft_consensus.cc:729] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa10776f19e643b889f94d6ab9107a97, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.178864 20593 tablet_bootstrap.cc:492] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: Bootstrap starting.
I20260812 14:42:29.178874 20597 tablet_bootstrap.cc:492] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: Bootstrap starting.
I20260812 14:42:29.178879 20601 consensus_queue.cc:260] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.178977 20601 ts_tablet_manager.cc:1428] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:29.179258 20597 tablet_bootstrap.cc:654] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.179498 20593 tablet_bootstrap.cc:654] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.179908 20597 tablet_bootstrap.cc:492] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: No bootstrap required, opened a new log
I20260812 14:42:29.179905 20611 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.179962 20597 ts_tablet_manager.cc:1397] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:29.179989 20611 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:29.180089 20597 raft_consensus.cc:348] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.180166 20597 raft_consensus.cc:374] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.180184 20597 raft_consensus.cc:729] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.180222 20597 consensus_queue.cc:260] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.180301 20597 ts_tablet_manager.cc:1428] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.180297 20593 tablet_bootstrap.cc:492] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: No bootstrap required, opened a new log
I20260812 14:42:29.180344 20593 ts_tablet_manager.cc:1397] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:29.180344 20609 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.180439 20609 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:29.180502 20593 raft_consensus.cc:348] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.180576 20593 raft_consensus.cc:374] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.180605 20593 raft_consensus.cc:729] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0eada9623c444afbd46294cab377db8, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.180658 20593 consensus_queue.cc:260] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.180764 20593 ts_tablet_manager.cc:1428] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.180962 20604 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.181054 20604 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:29.191557 20609 raft_consensus.cc:482] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:29.191632 20609 raft_consensus.cc:504] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.191752 20609 leader_election.cc:288] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855), aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235)
I20260812 14:42:29.191906 20462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9a5c2fe9ba431581eacb8fe95147fd" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa10776f19e643b889f94d6ab9107a97" is_pre_election: true
I20260812 14:42:29.191907 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9a5c2fe9ba431581eacb8fe95147fd" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8" is_pre_election: true
I20260812 14:42:29.191972 20462 raft_consensus.cc:2454] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 0.
I20260812 14:42:29.191972 20188 raft_consensus.cc:2454] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 0.
I20260812 14:42:29.192086 19755 leader_election.cc:302] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, aa10776f19e643b889f94d6ab9107a97; no voters:
I20260812 14:42:29.192178 20609 raft_consensus.cc:2781] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:29.192227 20609 raft_consensus.cc:482] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:29.192256 20609 raft_consensus.cc:3037] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.192816 20609 raft_consensus.cc:504] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.192925 20609 leader_election.cc:288] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 election: Requested vote from peers c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855), aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235)
I20260812 14:42:29.193037 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9a5c2fe9ba431581eacb8fe95147fd" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8"
I20260812 14:42:29.193096 20188 raft_consensus.cc:3037] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.193112 20462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf9a5c2fe9ba431581eacb8fe95147fd" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa10776f19e643b889f94d6ab9107a97"
I20260812 14:42:29.193181 20462 raft_consensus.cc:3037] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.193612 20188 raft_consensus.cc:2454] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 1.
I20260812 14:42:29.193718 20462 raft_consensus.cc:2454] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 1.
I20260812 14:42:29.193785 19755 leader_election.cc:302] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, c0eada9623c444afbd46294cab377db8; no voters:
I20260812 14:42:29.193864 20609 raft_consensus.cc:2781] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:29.193907 20609 raft_consensus.cc:686] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Running, Role: LEADER
I20260812 14:42:29.193943 20609 consensus_queue.cc:237] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.194195 20609 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: TxnStatusTablet state changed. Reason: New leader 8d1afc83f9a84a94b8bed6a748239727. Latest consensus state: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.194248 20609 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:29.194485 19583 catalog_manager.cc:5495] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193). New cstate: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:29.194758 20637 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:42:29.194880 20637 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:42:29.203496 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.201184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.15.227.193:33175)
0812 14:42:29.201233 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.201235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.201244 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:29.201323 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.201324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.201343 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.201407 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:29.201518 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.201991 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.202743 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.202939 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.202993 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.203195 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.203281 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.203351 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.203407 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.203429 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:42:29.203511 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.201122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33175 (local address 127.0.0.1:34784)
0812 14:42:29.201183 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.201194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.201242 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.201412 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.201413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.201414 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:29.201455 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.201457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.202109 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.202110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.202662 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.202664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.203011 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.203012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.203084 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.203103 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.203338 (+ 235us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.203346 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.203402 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.203446 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:42:29.205240 20462 raft_consensus.cc:1261] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Refusing update from remote peer 8d1afc83f9a84a94b8bed6a748239727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:29.205250 20188 raft_consensus.cc:1261] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Refusing update from remote peer 8d1afc83f9a84a94b8bed6a748239727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:29.205578 20617 consensus_queue.cc:1035] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:29.205715 20609 consensus_queue.cc:1035] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:29.206895 20611 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: TxnStatusTablet state changed. Reason: New leader 8d1afc83f9a84a94b8bed6a748239727. Latest consensus state: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.207002 20611 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:29.208179 20609 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.208254 20609 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:29.208698 20611 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.208913 20611 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P aa10776f19e643b889f94d6ab9107a97: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:29.210099 20604 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: TxnStatusTablet state changed. Reason: New leader 8d1afc83f9a84a94b8bed6a748239727. Latest consensus state: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.210199 20604 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:29.211040 20617 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.211125 20617 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P 8d1afc83f9a84a94b8bed6a748239727: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:29.211412 20645 mvcc.cc:205] Tried to move back new op lower bound from 7317691388742168576 to 7317691388698742784. Current Snapshot: MvccSnapshot[applied={T|T < 7317691388742168576}]
I20260812 14:42:29.211759 20604 tablet_replica.cc:439] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } } }
I20260812 14:42:29.211827 20604 tablet_replica.cc:442] T bf9a5c2fe9ba431581eacb8fe95147fd P c0eada9623c444afbd46294cab377db8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:29.214702 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.212417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.15.227.193:33175)
0812 14:42:29.212456 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.212457 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.212485 (+ 28us) server_negotiation.cc:407] Connection header received
0812 14:42:29.212570 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.212571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.212581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.212638 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:29.212766 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.213206 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.213998 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.214232 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.214288 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.214471 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.214504 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.214553 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.214619 (+ 66us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.214639 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:29.219380 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.216970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:35085 (local address 127.15.227.193:33175)
0812 14:42:29.217003 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.217005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.217038 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:42:29.217108 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.217108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.217116 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.217150 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:29.217269 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.217685 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.218722 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.218899 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.218960 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.219157 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.219183 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.219236 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.219297 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.219317 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":15}
I20260812 14:42:29.219451 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.216930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33175 (local address 127.15.227.193:35085)
0812 14:42:29.216997 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.217006 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.217049 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.217161 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.217162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.217163 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:29.217219 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.217220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.217753 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.217754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.218648 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.218651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.218972 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.218972 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.219035 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.219050 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.219245 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.219247 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.219295 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.219340 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":3}
I20260812 14:42:29.221448 20462 raft_consensus.cc:1261] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Refusing update from remote peer 8d1afc83f9a84a94b8bed6a748239727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:29.221489 20188 raft_consensus.cc:1261] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Refusing update from remote peer 8d1afc83f9a84a94b8bed6a748239727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:29.221645 20609 consensus_queue.cc:1035] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:29.221794 20655 consensus_queue.cc:1035] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:29.223078 20646 mvcc.cc:205] Tried to move back new op lower bound from 7317691388809773056 to 7317691388494950400. Current Snapshot: MvccSnapshot[applied={T|T < 7317691388809773056}]
I20260812 14:42:29.235234 19584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:29.235317 19584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:29.235962 19584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Sending DeleteTablet for 3 replicas of tablet 5fd9bf08fa4345b9a564432bf49fd89e
I20260812 14:42:29.236219 20145 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd9bf08fa4345b9a564432bf49fd89e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:29 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:52060
I20260812 14:42:29.236236 19862 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd9bf08fa4345b9a564432bf49fd89e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:29 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260812 14:42:29.236261 20416 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fd9bf08fa4345b9a564432bf49fd89e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:29 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55190
I20260812 14:42:29.236433 20677 tablet_replica.cc:330] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: stopping tablet replica
I20260812 14:42:29.236524 20677 raft_consensus.cc:2229] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:29.236574 20677 raft_consensus.cc:2258] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:29.236732 20682 tablet_replica.cc:330] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: stopping tablet replica
I20260812 14:42:29.236792 19584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:29.236927 19584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:29.237011 20677 ts_tablet_manager.cc:1905] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:29.236876 20682 raft_consensus.cc:2229] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:29.237203 20682 raft_consensus.cc:2258] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:29.237406 20679 tablet_replica.cc:330] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: stopping tablet replica
I20260812 14:42:29.237536 20679 raft_consensus.cc:2229] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:29.237675 20679 raft_consensus.cc:2258] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:29.237991 19862 tablet_service.cc:1467] Processing CreateTablet for tablet 052e483a44f64f0895b0ce28f8afca0d (DEFAULT_TABLE table=test-table [id=30464e31e69549b59bbe08ce392d199f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:29.238133 19862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052e483a44f64f0895b0ce28f8afca0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.238276 20679 ts_tablet_manager.cc:1905] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:29.238775 20677 ts_tablet_manager.cc:1918] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:29.238849 20677 log.cc:1196] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/5fd9bf08fa4345b9a564432bf49fd89e
I20260812 14:42:29.239092 20677 ts_tablet_manager.cc:1939] T 5fd9bf08fa4345b9a564432bf49fd89e P c0eada9623c444afbd46294cab377db8: Deleting consensus metadata
I20260812 14:42:29.239373 20597 tablet_bootstrap.cc:492] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Bootstrap starting.
I20260812 14:42:29.239780 20597 tablet_bootstrap.cc:654] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.239979 20679 ts_tablet_manager.cc:1918] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:29.240062 20679 log.cc:1196] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/5fd9bf08fa4345b9a564432bf49fd89e
I20260812 14:42:29.240288 19563 catalog_manager.cc:4841] TS c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855): tablet 5fd9bf08fa4345b9a564432bf49fd89e (table test-table [id=2584a55457eb4a26877b46e299fb60c6]) successfully deleted
I20260812 14:42:29.240326 20679 ts_tablet_manager.cc:1939] T 5fd9bf08fa4345b9a564432bf49fd89e P 8d1afc83f9a84a94b8bed6a748239727: Deleting consensus metadata
I20260812 14:42:29.240362 20145 tablet_service.cc:1467] Processing CreateTablet for tablet 052e483a44f64f0895b0ce28f8afca0d (DEFAULT_TABLE table=test-table [id=30464e31e69549b59bbe08ce392d199f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:29.240456 20145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052e483a44f64f0895b0ce28f8afca0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.240566 20416 tablet_service.cc:1467] Processing CreateTablet for tablet 052e483a44f64f0895b0ce28f8afca0d (DEFAULT_TABLE table=test-table [id=30464e31e69549b59bbe08ce392d199f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:29.240674 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 052e483a44f64f0895b0ce28f8afca0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:29.240654 19561 catalog_manager.cc:4841] TS 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175): tablet 5fd9bf08fa4345b9a564432bf49fd89e (table test-table [id=2584a55457eb4a26877b46e299fb60c6]) successfully deleted
I20260812 14:42:29.240986 20597 tablet_bootstrap.cc:492] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: No bootstrap required, opened a new log
I20260812 14:42:29.241024 20597 ts_tablet_manager.cc:1397] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:42:29.241154 20597 raft_consensus.cc:348] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.241206 20597 raft_consensus.cc:374] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.241225 20597 raft_consensus.cc:729] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.241263 20597 consensus_queue.cc:260] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.241341 20597 ts_tablet_manager.cc:1428] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.241560 20593 tablet_bootstrap.cc:492] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: Bootstrap starting.
W20260812 14:42:29.241712 20000 tablet.cc:2369] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:29.241920 20593 tablet_bootstrap.cc:654] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.241964 20601 tablet_bootstrap.cc:492] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Bootstrap starting.
I20260812 14:42:29.242347 20601 tablet_bootstrap.cc:654] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:29.242636 20593 tablet_bootstrap.cc:492] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: No bootstrap required, opened a new log
I20260812 14:42:29.242679 20593 ts_tablet_manager.cc:1397] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:29.242811 20593 raft_consensus.cc:348] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.242869 20593 raft_consensus.cc:374] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.242887 20593 raft_consensus.cc:729] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0eada9623c444afbd46294cab377db8, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.242925 20593 consensus_queue.cc:260] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.242942 20601 tablet_bootstrap.cc:492] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: No bootstrap required, opened a new log
I20260812 14:42:29.242985 20601 ts_tablet_manager.cc:1397] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:29.243002 20593 ts_tablet_manager.cc:1428] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.243129 20601 raft_consensus.cc:348] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.243187 20601 raft_consensus.cc:374] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:29.243206 20601 raft_consensus.cc:729] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa10776f19e643b889f94d6ab9107a97, State: Initialized, Role: FOLLOWER
I20260812 14:42:29.243249 20601 consensus_queue.cc:260] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.243255 20682 ts_tablet_manager.cc:1905] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:29.243346 20601 ts_tablet_manager.cc:1428] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:29.244520 20682 ts_tablet_manager.cc:1918] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:29.244581 20682 log.cc:1196] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/5fd9bf08fa4345b9a564432bf49fd89e
I20260812 14:42:29.244778 20682 ts_tablet_manager.cc:1939] T 5fd9bf08fa4345b9a564432bf49fd89e P aa10776f19e643b889f94d6ab9107a97: Deleting consensus metadata
I20260812 14:42:29.245096 19563 catalog_manager.cc:4841] TS aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235): tablet 5fd9bf08fa4345b9a564432bf49fd89e (table test-table [id=2584a55457eb4a26877b46e299fb60c6]) successfully deleted
W20260812 14:42:29.281985 20549 tablet.cc:2369] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:29.350859 20611 raft_consensus.cc:482] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:29.350970 20611 raft_consensus.cc:504] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.351311 20611 leader_election.cc:288] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175), c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855)
I20260812 14:42:29.354436 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.351483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33175 (local address 127.15.227.195:49933)
0812 14:42:29.351544 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.351556 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.351603 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.351806 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.351807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.351809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:29.351880 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.351882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.352780 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.352781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.353605 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.353608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.354161 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.354162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.354242 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.354246 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.354311 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.354347 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":43}
I20260812 14:42:29.354458 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.351585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:49933 (local address 127.15.227.193:33175)
0812 14:42:29.351629 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.351631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.351640 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:29.351680 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.351681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.351690 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.351753 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:29.351950 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.352699 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.353694 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.354081 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.354137 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.354149 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.354346 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.354389 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:42:29.354697 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052e483a44f64f0895b0ce28f8afca0d" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1afc83f9a84a94b8bed6a748239727" is_pre_election: true
I20260812 14:42:29.354852 19895 raft_consensus.cc:2454] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 0.
I20260812 14:42:29.355072 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.351483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34855 (local address 127.15.227.195:41285)
0812 14:42:29.351706 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.351714 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.351750 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.351907 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.351908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.351909 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:29.352248 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.352250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.352882 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.352884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.354119 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.354122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.354817 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.354818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.354887 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.354891 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.354969 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.355008 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":16,"thread_start_us":88,"threads_started":1}
I20260812 14:42:29.355049 20329 leader_election.cc:302] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, aa10776f19e643b889f94d6ab9107a97; no voters:
I20260812 14:42:29.355167 20611 raft_consensus.cc:2781] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:29.355196 20611 raft_consensus.cc:482] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:29.355221 20611 raft_consensus.cc:3037] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.355350 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.351766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:41285 (local address 127.15.227.194:34855)
0812 14:42:29.351810 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.351812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.351819 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:29.351841 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.351841 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.351848 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.351892 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:29.352322 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.352802 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.354200 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.354697 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.354761 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.354770 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.355248 (+ 478us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.355290 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:42:29.355525 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052e483a44f64f0895b0ce28f8afca0d" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8" is_pre_election: true
I20260812 14:42:29.355598 20188 raft_consensus.cc:2454] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 0.
I20260812 14:42:29.356065 20611 raft_consensus.cc:504] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.356199 20611 leader_election.cc:288] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 election: Requested vote from peers 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175), c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855)
I20260812 14:42:29.356369 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052e483a44f64f0895b0ce28f8afca0d" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8"
I20260812 14:42:29.356385 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "052e483a44f64f0895b0ce28f8afca0d" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1afc83f9a84a94b8bed6a748239727"
I20260812 14:42:29.356436 20188 raft_consensus.cc:3037] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.356465 19895 raft_consensus.cc:3037] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:29.357069 19895 raft_consensus.cc:2454] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 1.
I20260812 14:42:29.357152 20188 raft_consensus.cc:2454] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 1.
I20260812 14:42:29.357227 20329 leader_election.cc:302] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, aa10776f19e643b889f94d6ab9107a97; no voters:
I20260812 14:42:29.357314 20611 raft_consensus.cc:2781] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:29.357455 20611 raft_consensus.cc:686] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 1 LEADER]: Becoming Leader. State: Replica: aa10776f19e643b889f94d6ab9107a97, State: Running, Role: LEADER
I20260812 14:42:29.357527 20611 consensus_queue.cc:237] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:29.358229 19584 catalog_manager.cc:5495] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa10776f19e643b889f94d6ab9107a97 (127.15.227.195). New cstate: current_term: 1 leader_uuid: "aa10776f19e643b889f94d6ab9107a97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260812 14:42:29.393509 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.390912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.15.227.195:34235)
0812 14:42:29.390948 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.390950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.390974 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:42:29.391045 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.391046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.391063 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.391117 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:29.391244 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.391691 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.392715 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.392911 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.392973 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.393187 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.393294 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.393354 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.393435 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.393455 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:42:29.398479 20562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.395866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33175 (local address 127.15.227.195:42813)
0812 14:42:29.395929 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.395940 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.395978 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.396120 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.396121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.396122 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:29.396176 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.396177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.397039 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.397040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.397568 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.397570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.398070 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.398071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.398136 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.398152 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.398339 (+ 187us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.398342 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.398387 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.398431 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:42:29.398479 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.395954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:42813 (local address 127.15.227.193:33175)
0812 14:42:29.396000 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.396002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.396010 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:29.396033 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.396033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.396043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.396103 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:29.396239 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.396936 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.397663 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.397963 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.398070 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.398225 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.398261 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.398317 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.398392 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.398411 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:42:29.400484 20188 raft_consensus.cc:1261] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Refusing update from remote peer aa10776f19e643b889f94d6ab9107a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:29.400521 19895 raft_consensus.cc:1261] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Refusing update from remote peer aa10776f19e643b889f94d6ab9107a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:29.400735 20611 consensus_queue.cc:1035] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:29.400897 20692 consensus_queue.cc:1035] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:29.408264 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.405866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:53157 (local address 127.15.227.195:34235)
0812 14:42:29.405895 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:29.405896 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:29.405903 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:29.405921 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:29.405922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:29.405930 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:29.405972 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:29.406126 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.406822 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.407557 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.407835 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.407895 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.408004 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.408038 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.408086 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:29.408184 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 14:42:29.408203 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 14:42:29.408280 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:29.405784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:34235 (local address 127.15.227.193:53157)
0812 14:42:29.405824 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:29.405835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:29.405871 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:29.406010 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:29.406011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:29.406030 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:29.406068 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.406070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.406887 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.406888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.407485 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:29.407488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:29.407892 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:29.407893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:29.407930 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:29.407933 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:29.408134 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:29.408136 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:42:29.408174 (+ 38us) client_negotiation.cc:240] Negotiation successful
0812 14:42:29.408218 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
W20260812 14:42:29.408588 20646 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 14:42:30.928306 19583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:30.928409 19583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:30.929168 19583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Sending DeleteTablet for 3 replicas of tablet 052e483a44f64f0895b0ce28f8afca0d
I20260812 14:42:30.929390 20145 tablet_service.cc:1514] Processing DeleteTablet for tablet 052e483a44f64f0895b0ce28f8afca0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:30 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:52060
I20260812 14:42:30.929390 19862 tablet_service.cc:1514] Processing DeleteTablet for tablet 052e483a44f64f0895b0ce28f8afca0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:30 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260812 14:42:30.929565 20416 tablet_service.cc:1514] Processing DeleteTablet for tablet 052e483a44f64f0895b0ce28f8afca0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:30 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55190
I20260812 14:42:30.929749 20756 tablet_replica.cc:330] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: stopping tablet replica
I20260812 14:42:30.929818 20754 tablet_replica.cc:330] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: stopping tablet replica
I20260812 14:42:30.929836 20756 raft_consensus.cc:2229] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:30.929922 20756 raft_consensus.cc:2258] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:30.929920 20754 raft_consensus.cc:2229] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:30.929957 20754 raft_consensus.cc:2258] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:30.930228 20756 ts_tablet_manager.cc:1905] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:30.930276 20754 ts_tablet_manager.cc:1905] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:30.931886 20755 tablet_replica.cc:330] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: stopping tablet replica
I20260812 14:42:30.931947 20756 ts_tablet_manager.cc:1918] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:42:30.931993 20754 ts_tablet_manager.cc:1918] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:42:30.932026 20756 log.cc:1196] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/052e483a44f64f0895b0ce28f8afca0d
I20260812 14:42:30.932022 20755 raft_consensus.cc:2229] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:30.932111 20754 log.cc:1196] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/052e483a44f64f0895b0ce28f8afca0d
I20260812 14:42:30.932063 20755 raft_consensus.cc:2258] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:30.932276 20756 ts_tablet_manager.cc:1939] T 052e483a44f64f0895b0ce28f8afca0d P aa10776f19e643b889f94d6ab9107a97: Deleting consensus metadata
I20260812 14:42:30.932370 20754 ts_tablet_manager.cc:1939] T 052e483a44f64f0895b0ce28f8afca0d P 8d1afc83f9a84a94b8bed6a748239727: Deleting consensus metadata
I20260812 14:42:30.932461 20755 ts_tablet_manager.cc:1905] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:30.932611 19563 catalog_manager.cc:4841] TS aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235): tablet 052e483a44f64f0895b0ce28f8afca0d (table test-table [id=30464e31e69549b59bbe08ce392d199f]) successfully deleted
I20260812 14:42:30.932732 19561 catalog_manager.cc:4841] TS 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175): tablet 052e483a44f64f0895b0ce28f8afca0d (table test-table [id=30464e31e69549b59bbe08ce392d199f]) successfully deleted
I20260812 14:42:30.933624 20755 ts_tablet_manager.cc:1918] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:42:30.933673 20755 log.cc:1196] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/052e483a44f64f0895b0ce28f8afca0d
I20260812 14:42:30.933815 20755 ts_tablet_manager.cc:1939] T 052e483a44f64f0895b0ce28f8afca0d P c0eada9623c444afbd46294cab377db8: Deleting consensus metadata
I20260812 14:42:30.934181 19563 catalog_manager.cc:4841] TS c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855): tablet 052e483a44f64f0895b0ce28f8afca0d (table test-table [id=30464e31e69549b59bbe08ce392d199f]) successfully deleted
Aug 12 14:42:30 dist-test-slave-q9h9 krb5kdc[19501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545750, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:42:30 dist-test-slave-q9h9 krb5kdc[19501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545750, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:30.954345 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:30.947719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.15.227.254:37481)
0812 14:42:30.947849 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:30.947851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:30.947860 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:30.947950 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:30.947951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:30.947962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:30.948018 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:30.948564 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:30.949063 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:30.949768 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:30.949962 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:30.950035 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:30.952101 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:30.952102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:30.952102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:30.952109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:30.953353 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:30.953636 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:30.953636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:30.953637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:30.953686 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:30.953816 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:30.953817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:30.953817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:30.953916 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:30.953990 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:30.954170 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 14:42:30.954252 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260812 14:42:30.955698 19583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:30.955856 19583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:30.957193 20416 tablet_service.cc:1467] Processing CreateTablet for tablet 55d49d2e822d43ff9e5203a25c87f9ac (DEFAULT_TABLE table=test-table [id=638756d186524de8942139d1153b70e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:30.957194 20145 tablet_service.cc:1467] Processing CreateTablet for tablet 55d49d2e822d43ff9e5203a25c87f9ac (DEFAULT_TABLE table=test-table [id=638756d186524de8942139d1153b70e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:30.957319 20145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d49d2e822d43ff9e5203a25c87f9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:30.957325 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d49d2e822d43ff9e5203a25c87f9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:30.958137 19862 tablet_service.cc:1467] Processing CreateTablet for tablet 55d49d2e822d43ff9e5203a25c87f9ac (DEFAULT_TABLE table=test-table [id=638756d186524de8942139d1153b70e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:30.958274 19862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d49d2e822d43ff9e5203a25c87f9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:30.958434 20769 tablet_bootstrap.cc:492] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: Bootstrap starting.
I20260812 14:42:30.958465 20768 tablet_bootstrap.cc:492] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: Bootstrap starting.
I20260812 14:42:30.958818 20768 tablet_bootstrap.cc:654] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:30.958832 20769 tablet_bootstrap.cc:654] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:30.959342 20768 tablet_bootstrap.cc:492] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: No bootstrap required, opened a new log
I20260812 14:42:30.959393 20768 ts_tablet_manager.cc:1397] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:30.959444 20769 tablet_bootstrap.cc:492] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: No bootstrap required, opened a new log
I20260812 14:42:30.959486 20769 ts_tablet_manager.cc:1397] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:30.959514 20768 raft_consensus.cc:348] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.959580 20768 raft_consensus.cc:374] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:30.959600 20768 raft_consensus.cc:729] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa10776f19e643b889f94d6ab9107a97, State: Initialized, Role: FOLLOWER
I20260812 14:42:30.959635 20769 raft_consensus.cc:348] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.959654 20768 consensus_queue.cc:260] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.959695 20769 raft_consensus.cc:374] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:30.959714 20769 raft_consensus.cc:729] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0eada9623c444afbd46294cab377db8, State: Initialized, Role: FOLLOWER
I20260812 14:42:30.959733 20776 tablet_bootstrap.cc:492] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: Bootstrap starting.
I20260812 14:42:30.959758 20768 ts_tablet_manager.cc:1428] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:30.959767 20769 consensus_queue.cc:260] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.960122 20769 ts_tablet_manager.cc:1428] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:30.960209 20776 tablet_bootstrap.cc:654] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:30.960824 20776 tablet_bootstrap.cc:492] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: No bootstrap required, opened a new log
I20260812 14:42:30.960880 20776 ts_tablet_manager.cc:1397] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:30.960991 20776 raft_consensus.cc:348] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.961047 20776 raft_consensus.cc:374] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:30.961068 20776 raft_consensus.cc:729] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Initialized, Role: FOLLOWER
I20260812 14:42:30.961117 20776 consensus_queue.cc:260] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.961208 20776 ts_tablet_manager.cc:1428] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:30.997370 20725 raft_consensus.cc:482] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:30.997443 20725 raft_consensus.cc:504] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.997587 20725 leader_election.cc:288] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855), aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235)
I20260812 14:42:30.997756 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d49d2e822d43ff9e5203a25c87f9ac" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8" is_pre_election: true
I20260812 14:42:30.997774 20462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d49d2e822d43ff9e5203a25c87f9ac" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa10776f19e643b889f94d6ab9107a97" is_pre_election: true
I20260812 14:42:30.997849 20462 raft_consensus.cc:2454] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 0.
I20260812 14:42:30.997852 20188 raft_consensus.cc:2454] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 0.
I20260812 14:42:30.997975 19755 leader_election.cc:302] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, aa10776f19e643b889f94d6ab9107a97; no voters:
I20260812 14:42:30.998081 20725 raft_consensus.cc:2781] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:30.998127 20725 raft_consensus.cc:482] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:30.998140 20725 raft_consensus.cc:3037] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:30.998775 20725 raft_consensus.cc:504] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:30.998870 20725 leader_election.cc:288] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 election: Requested vote from peers c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855), aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235)
I20260812 14:42:30.999011 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d49d2e822d43ff9e5203a25c87f9ac" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8"
I20260812 14:42:30.999040 20462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d49d2e822d43ff9e5203a25c87f9ac" candidate_uuid: "8d1afc83f9a84a94b8bed6a748239727" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa10776f19e643b889f94d6ab9107a97"
I20260812 14:42:30.999076 20188 raft_consensus.cc:3037] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:30.999097 20462 raft_consensus.cc:3037] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:30.999631 20188 raft_consensus.cc:2454] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 1.
I20260812 14:42:30.999678 20462 raft_consensus.cc:2454] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1afc83f9a84a94b8bed6a748239727 in term 1.
I20260812 14:42:30.999800 19755 leader_election.cc:302] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d1afc83f9a84a94b8bed6a748239727, c0eada9623c444afbd46294cab377db8; no voters:
I20260812 14:42:30.999883 20725 raft_consensus.cc:2781] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:30.999938 20725 raft_consensus.cc:686] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 1 LEADER]: Becoming Leader. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Running, Role: LEADER
I20260812 14:42:30.999975 20725 consensus_queue.cc:237] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.000480 19584 catalog_manager.cc:5495] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193). New cstate: current_term: 1 leader_uuid: "8d1afc83f9a84a94b8bed6a748239727" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:31.009284 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:31.007312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34802 (local address 127.15.227.193:33175)
0812 14:42:31.007439 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:31.007441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:31.007447 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:42:31.007466 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:31.007466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:31.007476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:31.007531 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:31.007637 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:31.008121 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:31.008797 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:31.008989 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:31.009038 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:31.009119 (+ 81us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:31.009146 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:31.009177 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:31.009211 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 14:42:31.009228 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260812 14:42:31.011031 20188 raft_consensus.cc:1261] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Refusing update from remote peer 8d1afc83f9a84a94b8bed6a748239727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:31.011034 20462 raft_consensus.cc:1261] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Refusing update from remote peer 8d1afc83f9a84a94b8bed6a748239727: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:31.011225 20725 consensus_queue.cc:1035] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:31.011323 20715 consensus_queue.cc:1035] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:31.021823 19583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:31.021938 19583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:31.022562 19583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Sending DeleteTablet for 3 replicas of tablet 55d49d2e822d43ff9e5203a25c87f9ac
I20260812 14:42:31.022856 19861 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d49d2e822d43ff9e5203a25c87f9ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:31 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260812 14:42:31.022858 20416 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d49d2e822d43ff9e5203a25c87f9ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:31 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55190
I20260812 14:42:31.022977 20754 tablet_replica.cc:330] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: stopping tablet replica
I20260812 14:42:31.023041 20754 raft_consensus.cc:2229] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:31.023015 20145 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d49d2e822d43ff9e5203a25c87f9ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:31 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:52060
I20260812 14:42:31.023032 19583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 14:42:31.023103 20754 raft_consensus.cc:2258] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:31.023118 20755 tablet_replica.cc:330] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: stopping tablet replica
I20260812 14:42:31.023169 20755 raft_consensus.cc:2229] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:31.023177 20756 tablet_replica.cc:330] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: stopping tablet replica
W20260812 14:42:31.023185 19583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:31.023205 20755 raft_consensus.cc:2258] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:31.023231 20756 raft_consensus.cc:2229] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:31.023280 20756 raft_consensus.cc:2258] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:31.023594 20754 ts_tablet_manager.cc:1905] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:31.023638 20756 ts_tablet_manager.cc:1905] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:31.025068 20755 ts_tablet_manager.cc:1905] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:31.025112 20756 ts_tablet_manager.cc:1918] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:31.025180 20756 log.cc:1196] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/55d49d2e822d43ff9e5203a25c87f9ac
I20260812 14:42:31.025388 20756 ts_tablet_manager.cc:1939] T 55d49d2e822d43ff9e5203a25c87f9ac P aa10776f19e643b889f94d6ab9107a97: Deleting consensus metadata
I20260812 14:42:31.025636 20145 tablet_service.cc:1467] Processing CreateTablet for tablet 31de9c123c014e778d30bbc6155388ec (DEFAULT_TABLE table=test-table [id=ef5aafb37bf34c46b57224792ae9acad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:31.025683 20416 tablet_service.cc:1467] Processing CreateTablet for tablet 31de9c123c014e778d30bbc6155388ec (DEFAULT_TABLE table=test-table [id=ef5aafb37bf34c46b57224792ae9acad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:31.025755 20145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31de9c123c014e778d30bbc6155388ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:31.025792 20416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31de9c123c014e778d30bbc6155388ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:31.026583 20755 ts_tablet_manager.cc:1918] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:31.026629 20755 log.cc:1196] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/55d49d2e822d43ff9e5203a25c87f9ac
I20260812 14:42:31.026880 20755 ts_tablet_manager.cc:1939] T 55d49d2e822d43ff9e5203a25c87f9ac P c0eada9623c444afbd46294cab377db8: Deleting consensus metadata
I20260812 14:42:31.026926 20768 tablet_bootstrap.cc:492] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: Bootstrap starting.
I20260812 14:42:31.026947 20769 tablet_bootstrap.cc:492] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8: Bootstrap starting.
I20260812 14:42:31.027264 20769 tablet_bootstrap.cc:654] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:31.027271 20768 tablet_bootstrap.cc:654] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:31.027302 19861 tablet_service.cc:1467] Processing CreateTablet for tablet 31de9c123c014e778d30bbc6155388ec (DEFAULT_TABLE table=test-table [id=ef5aafb37bf34c46b57224792ae9acad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:31.027326 20754 ts_tablet_manager.cc:1918] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:31.027377 20754 log.cc:1196] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/55d49d2e822d43ff9e5203a25c87f9ac
I20260812 14:42:31.027423 19861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31de9c123c014e778d30bbc6155388ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:31.027575 20754 ts_tablet_manager.cc:1939] T 55d49d2e822d43ff9e5203a25c87f9ac P 8d1afc83f9a84a94b8bed6a748239727: Deleting consensus metadata
I20260812 14:42:31.027568 19563 catalog_manager.cc:4841] TS c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855): tablet 55d49d2e822d43ff9e5203a25c87f9ac (table test-table [id=638756d186524de8942139d1153b70e0]) successfully deleted
I20260812 14:42:31.027619 19563 catalog_manager.cc:4841] TS aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235): tablet 55d49d2e822d43ff9e5203a25c87f9ac (table test-table [id=638756d186524de8942139d1153b70e0]) successfully deleted
I20260812 14:42:31.027750 20768 tablet_bootstrap.cc:492] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: No bootstrap required, opened a new log
I20260812 14:42:31.027793 20768 ts_tablet_manager.cc:1397] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:31.027864 19561 catalog_manager.cc:4841] TS 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175): tablet 55d49d2e822d43ff9e5203a25c87f9ac (table test-table [id=638756d186524de8942139d1153b70e0]) successfully deleted
I20260812 14:42:31.027887 20768 raft_consensus.cc:348] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.027944 20768 raft_consensus.cc:374] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:31.027962 20768 raft_consensus.cc:729] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa10776f19e643b889f94d6ab9107a97, State: Initialized, Role: FOLLOWER
I20260812 14:42:31.027992 20769 tablet_bootstrap.cc:492] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8: No bootstrap required, opened a new log
I20260812 14:42:31.028002 20768 consensus_queue.cc:260] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.028031 20769 ts_tablet_manager.cc:1397] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:31.028086 20768 ts_tablet_manager.cc:1428] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:31.028155 20769 raft_consensus.cc:348] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.028225 20769 raft_consensus.cc:374] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:31.028246 20769 raft_consensus.cc:729] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0eada9623c444afbd46294cab377db8, State: Initialized, Role: FOLLOWER
I20260812 14:42:31.028280 20769 consensus_queue.cc:260] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.028364 20769 ts_tablet_manager.cc:1428] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:31.028937 20776 tablet_bootstrap.cc:492] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727: Bootstrap starting.
I20260812 14:42:31.029228 20776 tablet_bootstrap.cc:654] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:31.029775 20776 tablet_bootstrap.cc:492] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727: No bootstrap required, opened a new log
I20260812 14:42:31.029822 20776 ts_tablet_manager.cc:1397] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:31.029917 20776 raft_consensus.cc:348] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.029975 20776 raft_consensus.cc:374] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:31.029989 20776 raft_consensus.cc:729] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1afc83f9a84a94b8bed6a748239727, State: Initialized, Role: FOLLOWER
I20260812 14:42:31.030048 20776 consensus_queue.cc:260] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.030145 20776 ts_tablet_manager.cc:1428] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:31.248612 20737 raft_consensus.cc:482] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:31.248716 20737 raft_consensus.cc:504] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.248893 20737 leader_election.cc:288] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175), c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855)
I20260812 14:42:31.249176 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31de9c123c014e778d30bbc6155388ec" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8" is_pre_election: true
I20260812 14:42:31.249150 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31de9c123c014e778d30bbc6155388ec" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1afc83f9a84a94b8bed6a748239727" is_pre_election: true
I20260812 14:42:31.249269 19895 raft_consensus.cc:2454] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 0.
I20260812 14:42:31.249269 20188 raft_consensus.cc:2454] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 0.
I20260812 14:42:31.249432 20331 leader_election.cc:302] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa10776f19e643b889f94d6ab9107a97, c0eada9623c444afbd46294cab377db8; no voters:
I20260812 14:42:31.249557 20737 raft_consensus.cc:2781] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:31.249612 20737 raft_consensus.cc:482] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:31.249629 20737 raft_consensus.cc:3037] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:31.250465 20737 raft_consensus.cc:504] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.250607 20737 leader_election.cc:288] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 election: Requested vote from peers 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175), c0eada9623c444afbd46294cab377db8 (127.15.227.194:34855)
I20260812 14:42:31.250757 20188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31de9c123c014e778d30bbc6155388ec" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0eada9623c444afbd46294cab377db8"
I20260812 14:42:31.250825 20188 raft_consensus.cc:3037] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:31.250793 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31de9c123c014e778d30bbc6155388ec" candidate_uuid: "aa10776f19e643b889f94d6ab9107a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1afc83f9a84a94b8bed6a748239727"
I20260812 14:42:31.250878 19895 raft_consensus.cc:3037] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:31.251412 20188 raft_consensus.cc:2454] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 1.
I20260812 14:42:31.251513 19895 raft_consensus.cc:2454] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa10776f19e643b889f94d6ab9107a97 in term 1.
I20260812 14:42:31.251580 20331 leader_election.cc:302] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa10776f19e643b889f94d6ab9107a97, c0eada9623c444afbd46294cab377db8; no voters:
I20260812 14:42:31.251657 20737 raft_consensus.cc:2781] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:31.251701 20737 raft_consensus.cc:686] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 1 LEADER]: Becoming Leader. State: Replica: aa10776f19e643b889f94d6ab9107a97, State: Running, Role: LEADER
I20260812 14:42:31.251758 20737 consensus_queue.cc:237] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } }
I20260812 14:42:31.252346 19583 catalog_manager.cc:5495] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa10776f19e643b889f94d6ab9107a97 (127.15.227.195). New cstate: current_term: 1 leader_uuid: "aa10776f19e643b889f94d6ab9107a97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa10776f19e643b889f94d6ab9107a97" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34235 } health_report { overall_health: HEALTHY } } }
I20260812 14:42:31.313314 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:31.311258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.15.227.195:34235)
0812 14:42:31.311437 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:31.311438 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:31.311445 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:31.311464 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:31.311465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:31.311475 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:31.311528 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:31.311621 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:31.312150 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:31.312804 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:31.313008 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:31.313057 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:31.313136 (+ 79us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:31.313164 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:31.313196 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:31.313233 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 14:42:31.313254 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":123,"threads_started":1}
I20260812 14:42:31.315207 20188 raft_consensus.cc:1261] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Refusing update from remote peer aa10776f19e643b889f94d6ab9107a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:31.315296 19895 raft_consensus.cc:1261] T 31de9c123c014e778d30bbc6155388ec P 8d1afc83f9a84a94b8bed6a748239727 [term 1 FOLLOWER]: Refusing update from remote peer aa10776f19e643b889f94d6ab9107a97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:31.315435 20746 consensus_queue.cc:1035] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0eada9623c444afbd46294cab377db8" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:31.315546 20737 consensus_queue.cc:1035] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1afc83f9a84a94b8bed6a748239727" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:31.316936 20710 mvcc.cc:205] Tried to move back new op lower bound from 7317691397386072064 to 7317691397127696384. Current Snapshot: MvccSnapshot[applied={T|T < 7317691397386072064}]
W20260812 14:42:31.319700 20721 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 14:42:32.835805 19584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:32.835911 19584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:32.836658 19584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b380fb00152b4f968725ebbcd21c0546: Sending DeleteTablet for 3 replicas of tablet 31de9c123c014e778d30bbc6155388ec
I20260812 14:42:32.836936 19861 tablet_service.cc:1514] Processing DeleteTablet for tablet 31de9c123c014e778d30bbc6155388ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:32 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:34772
I20260812 14:42:32.836935 20416 tablet_service.cc:1514] Processing DeleteTablet for tablet 31de9c123c014e778d30bbc6155388ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:32 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55190
I20260812 14:42:32.836934 20145 tablet_service.cc:1514] Processing DeleteTablet for tablet 31de9c123c014e778d30bbc6155388ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:32 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:52060
I20260812 14:42:32.837025 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 19731
I20260812 14:42:32.842365 20856 tablet_replica.cc:330] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: stopping tablet replica
I20260812 14:42:32.842424 20856 raft_consensus.cc:2229] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:32.842476 20856 raft_consensus.cc:2258] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:32.844655 20856 ts_tablet_manager.cc:1905] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:32.847245 20856 ts_tablet_manager.cc:1918] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 14:42:32.847322 20856 log.cc:1196] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/31de9c123c014e778d30bbc6155388ec
I20260812 14:42:32.847538 20856 ts_tablet_manager.cc:1939] T 31de9c123c014e778d30bbc6155388ec P aa10776f19e643b889f94d6ab9107a97: Deleting consensus metadata
I20260812 14:42:32.847843 19563 catalog_manager.cc:4841] TS aa10776f19e643b889f94d6ab9107a97 (127.15.227.195:34235): tablet 31de9c123c014e778d30bbc6155388ec (table test-table [id=ef5aafb37bf34c46b57224792ae9acad]) successfully deleted
I20260812 14:42:32.852231 20855 tablet_replica.cc:330] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8: stopping tablet replica
I20260812 14:42:32.852286 20855 raft_consensus.cc:2229] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:32.852311 20855 raft_consensus.cc:2258] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:32.852514 20855 ts_tablet_manager.cc:1905] T 31de9c123c014e778d30bbc6155388ec P c0eada9623c444afbd46294cab377db8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260812 14:42:32.853005 19561 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:33175 (error 108)
I20260812 14:42:32.853303 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 20016
I20260812 14:42:32.882165 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:32.853371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33175 (local address 127.0.0.1:34814)
0812 14:42:32.882007 (+ 28636us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:32.882109 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28601,"thread_start_us":68,"threads_started":1}
W20260812 14:42:32.883088 19561 catalog_manager.cc:4584] TS 8d1afc83f9a84a94b8bed6a748239727 (127.15.227.193:33175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31de9c123c014e778d30bbc6155388ec: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33175: connect: Connection refused (error 111)
I20260812 14:42:32.895810 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:32.895711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34855 (local address 127.0.0.1:52064)
0812 14:42:32.895758 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:32.895780 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.194:34855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:42:32.895833 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 20296
I20260812 14:42:32.920828 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 19527
2026-08-12T14:42:32Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5768 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:32 dist-test-slave-q9h9 krb5kdc[20863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:32 dist-test-slave-q9h9 krb5kdc[20863](info): set up 2 sockets
Aug 12 14:42:32 dist-test-slave-q9h9 krb5kdc[20863](info): commencing operation
krb5kdc: starting...
W20260812 14:42:34.136682 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:34.245056 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:46355
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:46355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:34.324983 20892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:34.325182 20892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:34.325204 20892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:34.326308 20892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:34.326344 20892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:34.326357 20892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:34.326368 20892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:34.326381 20892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:34.327773 20892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:46355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:46355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:34.327955 20892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:34.328118 20892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:34.332680 20903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:34.332751 20902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:34.332683 20905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:34.332728 20892 server_base.cc:1034] running on GCE node
I20260812 14:42:34.333047 20892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:34.333304 20892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:34.334434 20892 hybrid_clock.cc:648] HybridClock initialized: now 1786545754334421 us; error 28 us; skew 500 ppm
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:34.336774 20892 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:34.337152 20892 webserver.cc:460] Webserver started at http://127.15.227.254:36679/ using document root <none> and password file <none>
I20260812 14:42:34.337335 20892 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:34.337383 20892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:34.337502 20892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:34.338506 20892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "281a5363399148b8b2144e9c0395d3b1"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "5e8864dbf4e1336c48357364b7407755"
server_key_iv: "10279a7a5fb6ebdc50fea14548c5248c"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.338824 20892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "281a5363399148b8b2144e9c0395d3b1"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "5e8864dbf4e1336c48357364b7407755"
server_key_iv: "10279a7a5fb6ebdc50fea14548c5248c"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.340080 20892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:34.340713 20914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.340863 20892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:34.340945 20892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "281a5363399148b8b2144e9c0395d3b1"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "5e8864dbf4e1336c48357364b7407755"
server_key_iv: "10279a7a5fb6ebdc50fea14548c5248c"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.341001 20892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:34.353461 20892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:34.355126 20892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:34.355252 20892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:34.358872 20892 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:46355
I20260812 14:42:34.358909 21019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:46355 every 8 connection(s)
I20260812 14:42:34.359202 20892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:34.359407 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 20892
I20260812 14:42:34.359493 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:34.359660 16271 external_mini_cluster.cc:1428] Setting key 74a24ef1decb1946621f594e9d6a5d7f
I20260812 14:42:34.360035 21022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:34.362555 21022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1: Bootstrap starting.
I20260812 14:42:34.363093 21022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:34.363344 21022 log.cc:824] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:34.364071 21022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1: No bootstrap required, opened a new log
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:34.365834 21022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281a5363399148b8b2144e9c0395d3b1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46355 } }
I20260812 14:42:34.365957 21022 raft_consensus.cc:374] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:34.365990 21022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 281a5363399148b8b2144e9c0395d3b1, State: Initialized, Role: FOLLOWER
I20260812 14:42:34.366133 21022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281a5363399148b8b2144e9c0395d3b1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46355 } }
I20260812 14:42:34.366197 21022 raft_consensus.cc:388] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:34.366255 21022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:34.366303 21022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:34.366791 21022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281a5363399148b8b2144e9c0395d3b1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46355 } }
I20260812 14:42:34.366882 21022 leader_election.cc:302] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 281a5363399148b8b2144e9c0395d3b1; no voters:
I20260812 14:42:34.367034 21022 leader_election.cc:288] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:34.367074 21029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:34.367233 21029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [term 1 LEADER]: Becoming Leader. State: Replica: 281a5363399148b8b2144e9c0395d3b1, State: Running, Role: LEADER
I20260812 14:42:34.367282 21022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:34.367430 21029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281a5363399148b8b2144e9c0395d3b1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46355 } }
I20260812 14:42:34.367746 21030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 281a5363399148b8b2144e9c0395d3b1. Latest consensus state: current_term: 1 leader_uuid: "281a5363399148b8b2144e9c0395d3b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281a5363399148b8b2144e9c0395d3b1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46355 } } }
I20260812 14:42:34.367864 21030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:34.368112 21046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:34.368607 21046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:34.369307 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:34.360114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.15.227.254:46355)
0812 14:42:34.360298 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:34.360305 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:34.360328 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:42:34.360712 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:34.360718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:34.360759 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:34.360876 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:34.361482 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.362375 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.363135 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.363625 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.363734 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:34.365825 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:34.365829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:34.365829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:34.365838 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:34.367614 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.367990 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.367993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.367993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.368098 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.368313 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.368314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.368315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.368631 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:34.368775 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:34.369017 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 14:42:34.369118 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":96,"threads_started":1}
I20260812 14:42:34.369326 21031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "281a5363399148b8b2144e9c0395d3b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "281a5363399148b8b2144e9c0395d3b1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46355 } } }
I20260812 14:42:34.369393 21031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:34.370764 21046 catalog_manager.cc:1347] Generated new cluster ID: 502815382b54499a8af3eabb010902f7
I20260812 14:42:34.370815 21046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:34.378895 21046 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:34.379357 21046 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:34.382820 21046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 281a5363399148b8b2144e9c0395d3b1: Generated new TSK 0
I20260812 14:42:34.382951 21046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:34.409409 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46355
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:34.488478 21064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:34.488754 21064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:34.488796 21064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:34.489928 21064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:34.489966 21064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:34.490001 21064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:34.491416 21064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:34.491629 21064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:34.491830 21064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:34.496569 21072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:34.496580 21074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:34.496579 21077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:34.496803 21064 server_base.cc:1034] running on GCE node
I20260812 14:42:34.496975 21064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:34.497195 21064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:34.498337 21064 hybrid_clock.cc:648] HybridClock initialized: now 1786545754498320 us; error 34 us; skew 500 ppm
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:34.500871 21064 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:34.501262 21064 webserver.cc:460] Webserver started at http://127.15.227.193:37649/ using document root <none> and password file <none>
I20260812 14:42:34.501451 21064 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:34.501492 21064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:34.501607 21064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:34.502615 21064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "86570a117b8c46938c842a32d254fe7b"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "9f3166a80d2cbe33bdd8679bfe0d7b84"
server_key_iv: "1eb8f77679e59058fcbabb7825de6bdf"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.502926 21064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "86570a117b8c46938c842a32d254fe7b"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "9f3166a80d2cbe33bdd8679bfe0d7b84"
server_key_iv: "1eb8f77679e59058fcbabb7825de6bdf"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.504161 21064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:34.504781 21091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.504923 21064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:34.505002 21064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "86570a117b8c46938c842a32d254fe7b"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "9f3166a80d2cbe33bdd8679bfe0d7b84"
server_key_iv: "1eb8f77679e59058fcbabb7825de6bdf"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.505075 21064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:34.512892 21064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:34.514467 21064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:34.514585 21064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:34.514776 21064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:34.515050 21064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:34.515082 21064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.515102 21064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:34.515134 21064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.520466 21064 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:37641
I20260812 14:42:34.520527 21299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:37641 every 8 connection(s)
I20260812 14:42:34.520779 21064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:34.523586 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21064
I20260812 14:42:34.523696 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:34.523869 16271 external_mini_cluster.cc:1428] Setting key b51b4c822706941997f24db1d42751ae
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:34.529639 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:34.521382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:48317 (local address 127.15.227.254:46355)
0812 14:42:34.521454 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:34.521456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:34.521599 (+ 143us) server_negotiation.cc:407] Connection header received
0812 14:42:34.522037 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:34.522039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:34.522051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:34.522114 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:34.522948 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.523499 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.524787 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.524986 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.525036 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:34.527197 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:34.527199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:34.527199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:34.527207 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:34.528428 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.528781 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.528782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.528783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.528836 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.529041 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.529042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.529043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.529146 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:34.529216 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:34.529463 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 14:42:34.529550 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:42:34.529780 21305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:34.521340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:46355 (local address 127.15.227.193:48317)
0812 14:42:34.521493 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:34.521526 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:34.521946 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:34.522183 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:34.522187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:34.522701 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:34.522881 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:34.522885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.523587 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.523588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:34.524710 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:34.524713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.525059 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.525060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:34.525201 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:34.525683 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:34.525693 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:34.527108 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:34.528502 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:34.528509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:34.528511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:34.528716 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:34.528889 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:34.528890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:34.528890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:34.528988 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:34.529218 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:34.529225 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:34.529351 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 14:42:34.529433 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:42:34.529573 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
I20260812 14:42:34.530221 21302 heartbeater.cc:347] Connected to a master server at 127.15.227.254:46355
I20260812 14:42:34.530323 21302 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:34.530506 21302 heartbeater.cc:511] Master 127.15.227.254:46355 requested a full tablet report, sending...
I20260812 14:42:34.530992 20949 ts_manager.cc:194] Registered new tserver with Master: 86570a117b8c46938c842a32d254fe7b (127.15.227.193:37641)
I20260812 14:42:34.531558 20949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:48317
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:34.546448 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46355
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:34.626286 21309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:34.626478 21309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:34.626499 21309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:34.627563 21309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:34.627595 21309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:34.627619 21309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:34.628962 21309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:34.629150 21309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:34.629316 21309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:42:34.633945 21309 server_base.cc:1034] running on GCE node
W20260812 14:42:34.633895 21314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:34.633909 21330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:34.633908 21315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:34.634336 21309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:34.634519 21309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:34.635679 21309 hybrid_clock.cc:648] HybridClock initialized: now 1786545754635660 us; error 29 us; skew 500 ppm
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:34.637938 21309 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:34.638334 21309 webserver.cc:460] Webserver started at http://127.15.227.194:41911/ using document root <none> and password file <none>
I20260812 14:42:34.638515 21309 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:34.638554 21309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:34.638645 21309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:34.639532 21309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "1649ff03e98e45afa7162d2e1b76ef76"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "930e45cd5dac7ba6b40e88c3262795cd"
server_key_iv: "600c5af8ea2647b40ba5ea25801c6d04"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.639822 21309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "1649ff03e98e45afa7162d2e1b76ef76"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "930e45cd5dac7ba6b40e88c3262795cd"
server_key_iv: "600c5af8ea2647b40ba5ea25801c6d04"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.640957 21309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:42:34.641585 21345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.641762 21309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.641840 21309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "1649ff03e98e45afa7162d2e1b76ef76"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "930e45cd5dac7ba6b40e88c3262795cd"
server_key_iv: "600c5af8ea2647b40ba5ea25801c6d04"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.641892 21309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:34.670799 21309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:34.672789 21309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:34.672900 21309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:34.673089 21309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:34.673386 21309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:34.673421 21309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.673441 21309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:34.673452 21309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.678864 21309 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:40651
I20260812 14:42:34.678963 21551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:40651 every 8 connection(s)
I20260812 14:42:34.679208 21309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:34.680985 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21309
I20260812 14:42:34.681089 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:34.681272 16271 external_mini_cluster.cc:1428] Setting key b9246fe77786518c9e24a2e90c0dbfe7
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:34.688100 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:34.679864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:38065 (local address 127.15.227.254:46355)
0812 14:42:34.679940 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:34.679942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:34.680134 (+ 192us) server_negotiation.cc:407] Connection header received
0812 14:42:34.680558 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:34.680559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:34.680570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:34.680620 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:34.681676 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.682266 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.683195 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.683397 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.683453 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:34.685555 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:34.685557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:34.685557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:34.685564 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:34.686803 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.687262 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.687263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.687263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.687307 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.687553 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.687554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.687554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.687639 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:34.687681 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:34.687982 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 14:42:34.688039 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 14:42:34.688349 21563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:34.679818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:46355 (local address 127.15.227.194:38065)
0812 14:42:34.679984 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:34.680024 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:34.680484 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:34.680671 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:34.680674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:34.681375 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:34.681595 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:34.681602 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.682351 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.682352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:34.683117 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:34.683119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.683469 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.683470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:34.683594 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:34.684009 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:34.684017 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:34.685475 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:34.686881 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:34.686890 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:34.686892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:34.687193 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:34.687364 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:34.687365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:34.687365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:34.687497 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:34.687691 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:34.687697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:34.687853 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 14:42:34.687970 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 14:42:34.688137 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260812 14:42:34.688843 21556 heartbeater.cc:347] Connected to a master server at 127.15.227.254:46355
I20260812 14:42:34.688935 21556 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:34.689116 21556 heartbeater.cc:511] Master 127.15.227.254:46355 requested a full tablet report, sending...
I20260812 14:42:34.689498 20949 ts_manager.cc:194] Registered new tserver with Master: 1649ff03e98e45afa7162d2e1b76ef76 (127.15.227.194:40651)
I20260812 14:42:34.689987 20949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:38065
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:34.708530 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46355
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:34.788338 21566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:34.788528 21566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:34.788549 21566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:34.789707 21566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:34.789749 21566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:34.789776 21566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:34.791218 21566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:34.791437 21566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:34.791640 21566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:34.796243 21576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:34.796275 21577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:34.796279 21566 server_base.cc:1034] running on GCE node
W20260812 14:42:34.796252 21583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:34.796622 21566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:34.796854 21566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:34.797977 21566 hybrid_clock.cc:648] HybridClock initialized: now 1786545754797965 us; error 30 us; skew 500 ppm
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:34.800231 21566 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:34.800613 21566 webserver.cc:460] Webserver started at http://127.15.227.195:38963/ using document root <none> and password file <none>
I20260812 14:42:34.800832 21566 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:34.800877 21566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:34.800995 21566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:34.801946 21566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "a616aa7dfe954e68b35d8c2081f5a4a1"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "7daeb060e3b4fce7ce5f3d86d744a3ad"
server_key_iv: "07a1e327805be2339588619a4e804105"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.802313 21566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "a616aa7dfe954e68b35d8c2081f5a4a1"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "7daeb060e3b4fce7ce5f3d86d744a3ad"
server_key_iv: "07a1e327805be2339588619a4e804105"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.803500 21566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:34.804137 21602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.804291 21566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:34.804368 21566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "a616aa7dfe954e68b35d8c2081f5a4a1"
format_stamp: "Formatted at 2026-08-12 14:42:34 on dist-test-slave-q9h9"
server_key: "7daeb060e3b4fce7ce5f3d86d744a3ad"
server_key_iv: "07a1e327805be2339588619a4e804105"
server_key_version: "encryptionkey@0"
I20260812 14:42:34.804431 21566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:34.814620 21566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:34.816231 21566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:34.816351 21566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:34.816530 21566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:34.816792 21566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:34.816824 21566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.816850 21566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:34.816875 21566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:34.822216 21566 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35503
I20260812 14:42:34.822292 21821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35503 every 8 connection(s)
I20260812 14:42:34.822536 21566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:42:34.822870 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21566
I20260812 14:42:34.822988 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:34.823202 16271 external_mini_cluster.cc:1428] Setting key 57849a4ac99ed6cde47517acfd6e8987
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[20863](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:34.830686 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:34.823167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:41923 (local address 127.15.227.254:46355)
0812 14:42:34.823209 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:34.823211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:34.823486 (+ 275us) server_negotiation.cc:407] Connection header received
0812 14:42:34.824021 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:34.824022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:34.824034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:34.824091 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:34.824910 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.825390 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.826348 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.826539 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.826587 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:34.828355 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:34.828356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:34.828357 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:34.828363 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:34.829540 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.829828 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.829829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.829829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.829874 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:34.830093 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:34.830095 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:34.830095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:34.830220 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:34.830286 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:34.830518 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 14:42:34.830599 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:42:34.830850 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:34.823161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:46355 (local address 127.15.227.195:41923)
0812 14:42:34.823349 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:34.823394 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:34.823934 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:34.824146 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:34.824149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:34.824668 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:34.824845 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:34.824849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.825450 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.825451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:34.826292 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:34.826294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:34.826593 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:34.826594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:34.826687 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:34.827048 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:34.827055 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:34.828280 (+ 1225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:34.829598 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:34.829601 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:34.829602 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:34.829780 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:34.829916 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:34.829916 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:34.829917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:34.829997 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:34.830283 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:34.830290 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:34.830405 (+ 115us) client_negotiation.cc:769] Sending connection context
0812 14:42:34.830487 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:42:34.830652 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260812 14:42:34.831238 21825 heartbeater.cc:347] Connected to a master server at 127.15.227.254:46355
I20260812 14:42:34.831319 21825 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:34.831471 21825 heartbeater.cc:511] Master 127.15.227.254:46355 requested a full tablet report, sending...
I20260812 14:42:34.831790 20949 ts_manager.cc:194] Registered new tserver with Master: a616aa7dfe954e68b35d8c2081f5a4a1 (127.15.227.195:35503)
I20260812 14:42:34.832136 20949 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:41923
I20260812 14:42:34.834723 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:42:34.846267 21027 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:34.845456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33022 (local address 127.15.227.254:46355)
0812 14:42:34.845508 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:34.845510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:34.845638 (+ 128us) server_negotiation.cc:407] Connection header received
0812 14:42:34.845680 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:34.845680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:34.845688 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:34.845741 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:34.846208 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33022: BlockingRecv error: recv got EOF from 127.0.0.1:33022 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:42:34.847260 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21064
I20260812 14:42:34.873001 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21309
I20260812 14:42:34.884680 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21566
I20260812 14:42:34.911509 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 20892
2026-08-12T14:42:34Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1987 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[21852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[21852](info): set up 2 sockets
Aug 12 14:42:34 dist-test-slave-q9h9 krb5kdc[21852](info): commencing operation
krb5kdc: starting...
W20260812 14:42:36.124922 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:36 dist-test-slave-q9h9 krb5kdc[21852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:36.234256 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:32989
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:32989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:36.315290 21877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:36.315488 21877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:36.315519 21877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:36.316655 21877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:36.316695 21877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:36.316715 21877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:36.316731 21877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:36.316747 21877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:36.318233 21877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:32989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:32989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:36.318459 21877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:36.318652 21877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:36.323091 21886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:36.323100 21887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:36.323163 21890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:36.323261 21877 server_base.cc:1034] running on GCE node
I20260812 14:42:36.323557 21877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:36.323838 21877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:36.324988 21877 hybrid_clock.cc:648] HybridClock initialized: now 1786545756324970 us; error 30 us; skew 500 ppm
Aug 12 14:42:36 dist-test-slave-q9h9 krb5kdc[21852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545756, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:36.327368 21877 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:36.327757 21877 webserver.cc:460] Webserver started at http://127.15.227.254:35653/ using document root <none> and password file <none>
I20260812 14:42:36.327942 21877 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:36.327988 21877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:36.328107 21877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:36.329089 21877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "d889c4f1e9744cff8d234443a4f056be"
format_stamp: "Formatted at 2026-08-12 14:42:36 on dist-test-slave-q9h9"
server_key: "7601fc72fe2e6dc350585d4fc61397fa"
server_key_iv: "bda66511e236b4a37925ecf927860258"
server_key_version: "encryptionkey@0"
I20260812 14:42:36.329414 21877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "d889c4f1e9744cff8d234443a4f056be"
format_stamp: "Formatted at 2026-08-12 14:42:36 on dist-test-slave-q9h9"
server_key: "7601fc72fe2e6dc350585d4fc61397fa"
server_key_iv: "bda66511e236b4a37925ecf927860258"
server_key_version: "encryptionkey@0"
I20260812 14:42:36.330682 21877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260812 14:42:36.331346 21902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:36.331487 21877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:36.331569 21877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "d889c4f1e9744cff8d234443a4f056be"
format_stamp: "Formatted at 2026-08-12 14:42:36 on dist-test-slave-q9h9"
server_key: "7601fc72fe2e6dc350585d4fc61397fa"
server_key_iv: "bda66511e236b4a37925ecf927860258"
server_key_version: "encryptionkey@0"
I20260812 14:42:36.331625 21877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:36.347106 21877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:36.349144 21877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:36.349284 21877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:36.353178 21877 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:32989
I20260812 14:42:36.353215 22003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:32989 every 8 connection(s)
I20260812 14:42:36.353616 21877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:36.354242 22004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:36.356352 22004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be: Bootstrap starting.
I20260812 14:42:36.356891 22004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:36.357158 22004 log.cc:824] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:36.357760 22004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be: No bootstrap required, opened a new log
I20260812 14:42:36.358814 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21877
I20260812 14:42:36.358892 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:36.359021 16271 external_mini_cluster.cc:1428] Setting key 5c2bd658d40447e97a727765ec39bdd0
I20260812 14:42:36.359529 22004 raft_consensus.cc:348] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d889c4f1e9744cff8d234443a4f056be" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32989 } }
I20260812 14:42:36.359660 22004 raft_consensus.cc:374] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:36.359694 22004 raft_consensus.cc:729] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d889c4f1e9744cff8d234443a4f056be, State: Initialized, Role: FOLLOWER
I20260812 14:42:36.359814 22004 consensus_queue.cc:260] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d889c4f1e9744cff8d234443a4f056be" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32989 } }
I20260812 14:42:36.359885 22004 raft_consensus.cc:388] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:36.359915 22004 raft_consensus.cc:482] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:36.359964 22004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:36.360606 22004 raft_consensus.cc:504] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d889c4f1e9744cff8d234443a4f056be" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32989 } }
I20260812 14:42:36.360699 22004 leader_election.cc:302] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d889c4f1e9744cff8d234443a4f056be; no voters:
I20260812 14:42:36.360852 22004 leader_election.cc:288] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:36.360883 22014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:36.361016 22014 raft_consensus.cc:686] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [term 1 LEADER]: Becoming Leader. State: Replica: d889c4f1e9744cff8d234443a4f056be, State: Running, Role: LEADER
I20260812 14:42:36.361091 22004 sys_catalog.cc:564] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:36.361171 22014 consensus_queue.cc:237] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d889c4f1e9744cff8d234443a4f056be" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32989 } }
I20260812 14:42:36.361514 22019 sys_catalog.cc:455] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d889c4f1e9744cff8d234443a4f056be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d889c4f1e9744cff8d234443a4f056be" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32989 } } }
I20260812 14:42:36.361589 22019 sys_catalog.cc:458] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:36.361729 22018 sys_catalog.cc:455] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [sys.catalog]: SysCatalogTable state changed. Reason: New leader d889c4f1e9744cff8d234443a4f056be. Latest consensus state: current_term: 1 leader_uuid: "d889c4f1e9744cff8d234443a4f056be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d889c4f1e9744cff8d234443a4f056be" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32989 } } }
I20260812 14:42:36.361783 22018 sys_catalog.cc:458] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:36.362172 22028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:36.362703 22028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:36.365072 22028 catalog_manager.cc:1347] Generated new cluster ID: 640a305eca394551956a1adf7db3dd31
I20260812 14:42:36.365123 22028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:42:36 dist-test-slave-q9h9 krb5kdc[21852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:36.369884 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:36.359437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.15.227.254:32989)
0812 14:42:36.360026 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:36.360034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:36.360066 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:42:36.360192 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:36.360199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:36.360268 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:36.360412 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:36.361017 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:36.362330 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:36.363365 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:36.363966 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:36.364075 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:36.366616 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:36.366620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:36.366620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:36.366631 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:36.368378 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:36.368768 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:36.368770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:36.368771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:36.368873 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:36.369074 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:36.369075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:36.369075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:36.369328 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:36.369458 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:36.369572 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 14:42:36.369693 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":492,"threads_started":1}
I20260812 14:42:36.370806 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:42:36.375777 22028 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:36.376366 22028 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:36.385282 22028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d889c4f1e9744cff8d234443a4f056be: Generated new TSK 0
I20260812 14:42:36.385435 22028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:42:36.388391 22012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:36.384827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49202 (local address 127.15.227.254:32989)
0812 14:42:36.384885 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:36.384888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:36.384996 (+ 108us) server_negotiation.cc:407] Connection header received
0812 14:42:36.385067 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:36.385068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:36.385087 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:36.385132 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:36.385544 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:36.386349 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:36.387032 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:36.387360 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:36.387426 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:36.387973 (+ 547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:36.387975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:36.387975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:36.388172 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:36.388190 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:36.388197 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:36.388255 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:36.388295 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 14:42:36.388317 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:42:36.390725 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21877
2026-08-12T14:42:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1483 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:36 dist-test-slave-q9h9 krb5kdc[22075](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:36 dist-test-slave-q9h9 krb5kdc[22075](info): set up 2 sockets
Aug 12 14:42:36 dist-test-slave-q9h9 krb5kdc[22075](info): commencing operation
krb5kdc: starting...
W20260812 14:42:37.575745 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:37 dist-test-slave-q9h9 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:37.683571 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:41869
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:40043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:41869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:37.762140 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:37.762319 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:37.762351 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:37.763445 22097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:37.763484 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:37.763502 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:37.763518 22097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:37.763535 22097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:37.764966 22097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:41869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:41869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:37.765178 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:37.765383 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:37.769868 22105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:37.770069 22108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:37.770156 22097 server_base.cc:1034] running on GCE node
W20260812 14:42:37.770262 22112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:37.770520 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:37.770779 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:37.771924 22097 hybrid_clock.cc:648] HybridClock initialized: now 1786545757771907 us; error 32 us; skew 500 ppm
Aug 12 14:42:37 dist-test-slave-q9h9 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545757, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:37.774207 22097 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:37.774601 22097 webserver.cc:460] Webserver started at http://127.15.227.254:46621/ using document root <none> and password file <none>
I20260812 14:42:37.774827 22097 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:37.774873 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:37.774994 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:37.775950 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "31a5c479047b4d479cebd540bf1dabcd"
format_stamp: "Formatted at 2026-08-12 14:42:37 on dist-test-slave-q9h9"
server_key: "bc002e8ae52f8e030f42a454869d4533"
server_key_iv: "8fb7e65693fd26dfd823cc22ef2fc5ed"
server_key_version: "encryptionkey@0"
I20260812 14:42:37.776280 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "31a5c479047b4d479cebd540bf1dabcd"
format_stamp: "Formatted at 2026-08-12 14:42:37 on dist-test-slave-q9h9"
server_key: "bc002e8ae52f8e030f42a454869d4533"
server_key_iv: "8fb7e65693fd26dfd823cc22ef2fc5ed"
server_key_version: "encryptionkey@0"
I20260812 14:42:37.777451 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:37.778088 22127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:37.778228 22097 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:37.778311 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "31a5c479047b4d479cebd540bf1dabcd"
format_stamp: "Formatted at 2026-08-12 14:42:37 on dist-test-slave-q9h9"
server_key: "bc002e8ae52f8e030f42a454869d4533"
server_key_iv: "8fb7e65693fd26dfd823cc22ef2fc5ed"
server_key_version: "encryptionkey@0"
I20260812 14:42:37.778376 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:37.791244 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:37.792709 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:37.792837 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:37.796528 22097 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:41869
I20260812 14:42:37.796571 22236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:41869 every 8 connection(s)
I20260812 14:42:37.796859 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:37.797400 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 22097
I20260812 14:42:37.797497 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:37.797659 16271 external_mini_cluster.cc:1428] Setting key 962a04a0cf05a42925688e7eacb76f19
I20260812 14:42:37.797943 22239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:37.800781 22239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd: Bootstrap starting.
I20260812 14:42:37.801434 22239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:37.801752 22239 log.cc:824] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:37.802451 22239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd: No bootstrap required, opened a new log
I20260812 14:42:37.804502 22239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a5c479047b4d479cebd540bf1dabcd" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41869 } }
I20260812 14:42:37.804625 22239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:37.804647 22239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31a5c479047b4d479cebd540bf1dabcd, State: Initialized, Role: FOLLOWER
I20260812 14:42:37.804749 22239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a5c479047b4d479cebd540bf1dabcd" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41869 } }
I20260812 14:42:37.804829 22239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:37.804858 22239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:37.804903 22239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 0 FOLLOWER]: Advancing to term 1
Aug 12 14:42:37 dist-test-slave-q9h9 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:37.805536 22239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a5c479047b4d479cebd540bf1dabcd" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41869 } }
I20260812 14:42:37.805634 22239 leader_election.cc:302] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31a5c479047b4d479cebd540bf1dabcd; no voters:
I20260812 14:42:37.805778 22239 leader_election.cc:288] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:37.805809 22261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:37.805979 22261 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [term 1 LEADER]: Becoming Leader. State: Replica: 31a5c479047b4d479cebd540bf1dabcd, State: Running, Role: LEADER
I20260812 14:42:37.806036 22239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:37.806164 22261 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a5c479047b4d479cebd540bf1dabcd" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41869 } }
I20260812 14:42:37.806615 22262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31a5c479047b4d479cebd540bf1dabcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a5c479047b4d479cebd540bf1dabcd" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41869 } } }
I20260812 14:42:37.806604 22261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31a5c479047b4d479cebd540bf1dabcd. Latest consensus state: current_term: 1 leader_uuid: "31a5c479047b4d479cebd540bf1dabcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31a5c479047b4d479cebd540bf1dabcd" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41869 } } }
I20260812 14:42:37.806775 22261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:37.806761 22262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:37.807077 22280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:37.807788 22280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:37.809973 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:37.798136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54860 (local address 127.15.227.254:41869)
0812 14:42:37.798328 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:37.798336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:37.798367 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:42:37.800089 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:37.800099 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:37.800159 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:37.800296 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:37.800865 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:37.802182 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:37.802946 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:37.803546 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:37.803666 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:37.805946 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:37.805950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:37.805951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:37.805962 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:37.808271 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:37.808694 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:37.808697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:37.808697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:37.808780 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:37.808971 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:37.808973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:37.808973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:37.809260 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:37.809411 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:37.809635 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 14:42:37.809773 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":83,"threads_started":1}
I20260812 14:42:37.810575 22280 catalog_manager.cc:1347] Generated new cluster ID: 2aadd9f7f88a45fcbf1be02a093837d8
I20260812 14:42:37.810624 22280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:37.821225 22280 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:37.821686 22280 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:37.826936 22280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31a5c479047b4d479cebd540bf1dabcd: Generated new TSK 0
I20260812 14:42:37.827076 22280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:37.852659 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41869
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:40043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:37.932242 22304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:37.932435 22304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:37.932456 22304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:37.933568 22304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:37.933609 22304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:37.933633 22304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:37.935104 22304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:37.935307 22304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:37.935511 22304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:37.940299 22319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:37.940498 22304 server_base.cc:1034] running on GCE node
W20260812 14:42:37.940593 22320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:37.940676 22326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:37.940882 22304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:37.941080 22304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:37.942217 22304 hybrid_clock.cc:648] HybridClock initialized: now 1786545757942207 us; error 34 us; skew 500 ppm
Aug 12 14:42:37 dist-test-slave-q9h9 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545757, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:37.944628 22304 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:37.945065 22304 webserver.cc:460] Webserver started at http://127.15.227.193:44343/ using document root <none> and password file <none>
I20260812 14:42:37.945262 22304 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:37.945309 22304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:37.945436 22304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:37.946482 22304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "2f3f31fba3084f3bb6c0a7719a677e18"
format_stamp: "Formatted at 2026-08-12 14:42:37 on dist-test-slave-q9h9"
server_key: "c5443fe2a3f513cb9c1ad538cdfd1adb"
server_key_iv: "5f12f369391201645f4c0bf8ec2e7bf5"
server_key_version: "encryptionkey@0"
I20260812 14:42:37.946827 22304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "2f3f31fba3084f3bb6c0a7719a677e18"
format_stamp: "Formatted at 2026-08-12 14:42:37 on dist-test-slave-q9h9"
server_key: "c5443fe2a3f513cb9c1ad538cdfd1adb"
server_key_iv: "5f12f369391201645f4c0bf8ec2e7bf5"
server_key_version: "encryptionkey@0"
I20260812 14:42:37.948016 22304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:37.948657 22346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:37.948824 22304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:37.948905 22304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "2f3f31fba3084f3bb6c0a7719a677e18"
format_stamp: "Formatted at 2026-08-12 14:42:37 on dist-test-slave-q9h9"
server_key: "c5443fe2a3f513cb9c1ad538cdfd1adb"
server_key_iv: "5f12f369391201645f4c0bf8ec2e7bf5"
server_key_version: "encryptionkey@0"
I20260812 14:42:37.948966 22304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:37.965406 22304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:37.967324 22304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:37.967447 22304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:37.967640 22304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:37.967926 22304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:37.967958 22304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:37.967985 22304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:37.968008 22304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:37.973278 22304 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:32861
I20260812 14:42:37.973335 22552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:32861 every 8 connection(s)
I20260812 14:42:37.973600 22304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:37.976900 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 22304
I20260812 14:42:37.976991 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:37.977165 16271 external_mini_cluster.cc:1428] Setting key ef6e15c889df39e1b630ff12e7d730f1
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:37.998343 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41869
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:40043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:42:37 dist-test-slave-q9h9 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545757, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:38.003463 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:37.974229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:37467 (local address 127.15.227.254:41869)
0812 14:42:37.974303 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:37.974306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:37.974478 (+ 172us) server_negotiation.cc:407] Connection header received
0812 14:42:37.974943 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:37.974944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:37.974954 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:37.974997 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:37.975837 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:37.976324 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:37.977371 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:37.977664 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:37.977730 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:38.000813 (+ 23083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:38.000815 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:38.000815 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:38.000823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:38.002167 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.002566 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.002568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.002568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.002624 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.002852 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.002853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.002853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.002967 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:38.003022 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:38.003284 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 14:42:38.003373 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:42:38.003638 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:37.974205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41869 (local address 127.15.227.193:37467)
0812 14:42:37.974369 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:37.974403 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:37.974870 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:37.975055 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:37.975058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:37.975584 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:37.975772 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:37.975776 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:37.976424 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:37.976426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:37.977289 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:37.977292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:37.977748 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:37.977750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:37.977911 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:37.999404 (+ 21493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:37.999414 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:38.000709 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:38.002254 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:38.002262 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:38.002272 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:38.002496 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:38.002680 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:38.002680 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:38.002681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:38.002795 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:38.003022 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:38.003029 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:38.003165 (+ 136us) client_negotiation.cc:769] Sending connection context
0812 14:42:38.003255 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 14:42:38.003415 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260812 14:42:38.004122 22555 heartbeater.cc:347] Connected to a master server at 127.15.227.254:41869
I20260812 14:42:38.004221 22555 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:38.004405 22555 heartbeater.cc:511] Master 127.15.227.254:41869 requested a full tablet report, sending...
I20260812 14:42:38.004980 22162 ts_manager.cc:194] Registered new tserver with Master: 2f3f31fba3084f3bb6c0a7719a677e18 (127.15.227.193:32861)
I20260812 14:42:38.005904 22162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:37467
W20260812 14:42:38.078912 22560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:38.079095 22560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:38.079116 22560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:38.080165 22560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:38.080199 22560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:38.080222 22560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:38.081676 22560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:38.081859 22560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:38.082082 22560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:38.086679 22572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:38.086688 22568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:38.086688 22567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:38.086812 22560 server_base.cc:1034] running on GCE node
I20260812 14:42:38.087057 22560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:38.087281 22560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:38.088413 22560 hybrid_clock.cc:648] HybridClock initialized: now 1786545758088402 us; error 24 us; skew 500 ppm
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:38.090826 22560 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:38.091249 22560 webserver.cc:460] Webserver started at http://127.15.227.194:33705/ using document root <none> and password file <none>
I20260812 14:42:38.091444 22560 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:38.091490 22560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:38.091612 22560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:38.092586 22560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "22e28fcde1fb45dabe5a3971defa1385"
format_stamp: "Formatted at 2026-08-12 14:42:38 on dist-test-slave-q9h9"
server_key: "5cfc4c03f68544fd72e75edd30edf8f1"
server_key_iv: "d7f684358f804ec177482f5f7fbe2c9c"
server_key_version: "encryptionkey@0"
I20260812 14:42:38.092906 22560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "22e28fcde1fb45dabe5a3971defa1385"
format_stamp: "Formatted at 2026-08-12 14:42:38 on dist-test-slave-q9h9"
server_key: "5cfc4c03f68544fd72e75edd30edf8f1"
server_key_iv: "d7f684358f804ec177482f5f7fbe2c9c"
server_key_version: "encryptionkey@0"
I20260812 14:42:38.094228 22560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:42:38.094897 22580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:38.095047 22560 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:38.095153 22560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "22e28fcde1fb45dabe5a3971defa1385"
format_stamp: "Formatted at 2026-08-12 14:42:38 on dist-test-slave-q9h9"
server_key: "5cfc4c03f68544fd72e75edd30edf8f1"
server_key_iv: "d7f684358f804ec177482f5f7fbe2c9c"
server_key_version: "encryptionkey@0"
I20260812 14:42:38.095220 22560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:38.105026 22560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:38.107024 22560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:38.107182 22560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:38.107383 22560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:38.107661 22560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:38.107702 22560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:38.107730 22560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:38.107762 22560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:38.113145 22560 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:38953
I20260812 14:42:38.113186 22789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:38953 every 8 connection(s)
I20260812 14:42:38.113454 22560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:38.121695 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:38.114068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:56061 (local address 127.15.227.254:41869)
0812 14:42:38.114116 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:38.114118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:38.114316 (+ 198us) server_negotiation.cc:407] Connection header received
0812 14:42:38.114813 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:38.114815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:38.114829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:38.114882 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:38.115752 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.116247 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.117189 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.117381 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.117436 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:38.119415 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:38.119416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:38.119416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:38.119424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:38.120479 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.120829 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.120830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.120831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.120884 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.121090 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.121091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.121091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.121201 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:38.121258 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:38.121500 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 14:42:38.121592 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:42:38.121805 22799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:38.114050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41869 (local address 127.15.227.194:56061)
0812 14:42:38.114207 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:38.114240 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:38.114704 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:38.114947 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:38.114950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:38.115485 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:38.115663 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:38.115668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.116327 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.116329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:38.117102 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:38.117104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.117449 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.117450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:38.117560 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:38.117987 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:38.117995 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:38.119333 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:38.120550 (+ 1217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:38.120557 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:38.120559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:38.120764 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:38.120938 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:38.120939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:38.120939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:38.121034 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:38.121258 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:38.121265 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:38.121386 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 14:42:38.121470 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:42:38.121614 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260812 14:42:38.122251 22790 heartbeater.cc:347] Connected to a master server at 127.15.227.254:41869
I20260812 14:42:38.122330 22790 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:38.122478 22790 heartbeater.cc:511] Master 127.15.227.254:41869 requested a full tablet report, sending...
I20260812 14:42:38.122555 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 22560
I20260812 14:42:38.122659 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:38.122828 16271 external_mini_cluster.cc:1428] Setting key 76d66629dcaf6ed758cd74f71ac7d2db
I20260812 14:42:38.122859 22162 ts_manager.cc:194] Registered new tserver with Master: 22e28fcde1fb45dabe5a3971defa1385 (127.15.227.194:38953)
I20260812 14:42:38.123306 22162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:56061
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:38.144824 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41869
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:40043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:38.224052 22805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:38.224251 22805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:38.224282 22805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:38.225385 22805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:38.225426 22805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:38.225461 22805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:38.226910 22805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:38.227149 22805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:38.227349 22805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:38.232245 22825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:38.232273 22816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:38.232445 22805 server_base.cc:1034] running on GCE node
W20260812 14:42:38.232280 22820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:38.232685 22805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:38.232904 22805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:38.234054 22805 hybrid_clock.cc:648] HybridClock initialized: now 1786545758234043 us; error 30 us; skew 500 ppm
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:38.236937 22805 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:38.237356 22805 webserver.cc:460] Webserver started at http://127.15.227.195:41459/ using document root <none> and password file <none>
I20260812 14:42:38.237612 22805 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:38.237677 22805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:38.237797 22805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:38.238891 22805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "92b74873a27645b19e92fb615f693ba9"
format_stamp: "Formatted at 2026-08-12 14:42:38 on dist-test-slave-q9h9"
server_key: "296c9edce8e8aaeb8d570aae7118531f"
server_key_iv: "48c6873703913cd5b6da39e7c21d1dd7"
server_key_version: "encryptionkey@0"
I20260812 14:42:38.239190 22805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "92b74873a27645b19e92fb615f693ba9"
format_stamp: "Formatted at 2026-08-12 14:42:38 on dist-test-slave-q9h9"
server_key: "296c9edce8e8aaeb8d570aae7118531f"
server_key_iv: "48c6873703913cd5b6da39e7c21d1dd7"
server_key_version: "encryptionkey@0"
I20260812 14:42:38.240324 22805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:38.240928 22837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:38.241055 22805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:38.241165 22805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "92b74873a27645b19e92fb615f693ba9"
format_stamp: "Formatted at 2026-08-12 14:42:38 on dist-test-slave-q9h9"
server_key: "296c9edce8e8aaeb8d570aae7118531f"
server_key_iv: "48c6873703913cd5b6da39e7c21d1dd7"
server_key_version: "encryptionkey@0"
I20260812 14:42:38.241232 22805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:38.249611 22805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:38.251237 22805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:38.251360 22805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:38.251544 22805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:38.251857 22805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:38.251891 22805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:38.251910 22805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:38.251933 22805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:38.257335 22805 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:37325
I20260812 14:42:38.257391 23057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:37325 every 8 connection(s)
I20260812 14:42:38.257655 22805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:42:38.259110 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 22805
I20260812 14:42:38.259203 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:38.259371 16271 external_mini_cluster.cc:1428] Setting key 0346b4f6c2c280c1a77d20845b327935
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:38.265997 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:38.258319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:51423 (local address 127.15.227.254:41869)
0812 14:42:38.258363 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:38.258365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:38.258531 (+ 166us) server_negotiation.cc:407] Connection header received
0812 14:42:38.258973 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:38.258975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:38.258987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:38.259059 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:38.260053 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.260593 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.261538 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.261724 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.261783 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:38.263693 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:38.263695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:38.263695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:38.263701 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:38.264931 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.265308 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.265309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.265310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.265357 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.265546 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.265547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.265547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.265633 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:38.265676 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:38.265883 (+ 207us) server_negotiation.cc:299] Negotiation successful
0812 14:42:38.265937 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:38.266217 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:38.258272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41869 (local address 127.15.227.195:51423)
0812 14:42:38.258421 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:38.258459 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:38.258895 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:38.259115 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:38.259118 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:38.259733 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:38.259986 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:38.259992 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.260669 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.260670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:38.261458 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:38.261460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.261812 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.261814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:38.261920 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:38.262358 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:38.262365 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:38.263618 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:38.265016 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:38.265022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:38.265024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:38.265241 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:38.265401 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:38.265401 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:38.265401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:38.265502 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:38.265676 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:38.265680 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:38.265785 (+ 105us) client_negotiation.cc:769] Sending connection context
0812 14:42:38.265862 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:42:38.265974 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260812 14:42:38.266635 23058 heartbeater.cc:347] Connected to a master server at 127.15.227.254:41869
I20260812 14:42:38.266722 23058 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:38.266884 23058 heartbeater.cc:511] Master 127.15.227.254:41869 requested a full tablet report, sending...
I20260812 14:42:38.267222 22161 ts_manager.cc:194] Registered new tserver with Master: 92b74873a27645b19e92fb615f693ba9 (127.15.227.195:37325)
I20260812 14:42:38.267578 22161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:51423
I20260812 14:42:38.270908 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[22075](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545758, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[22075](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545758, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:38.293385 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:38.287192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.15.227.254:41869)
0812 14:42:38.287238 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:38.287240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:38.287317 (+ 77us) server_negotiation.cc:407] Connection header received
0812 14:42:38.287383 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:38.287384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:38.287392 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:38.287428 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:38.287942 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.288398 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.289074 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:38.289257 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:38.289303 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:38.291286 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:38.291288 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:38.291288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:38.291295 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:38.292462 (+ 1167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.292760 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.292761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.292761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.292804 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:38.292941 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:38.292942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:38.292942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:38.293059 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:38.293130 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:38.293269 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 14:42:38.293322 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260812 14:42:38.293620 22161 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42584
I20260812 14:42:38.293993 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 22304
I20260812 14:42:38.309592 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 22560
I20260812 14:42:38.325352 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 22805
I20260812 14:42:38.340714 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 22097
2026-08-12T14:42:38Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1946 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[23077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[23077](info): set up 2 sockets
Aug 12 14:42:38 dist-test-slave-q9h9 krb5kdc[23077](info): commencing operation
krb5kdc: starting...
W20260812 14:42:39.524952 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:39 dist-test-slave-q9h9 krb5kdc[23077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:39.634588 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:46359
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:46359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:39.715663 23102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:39.716079 23102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:39.716168 23102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:39.717686 23102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:39.717731 23102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:39.717751 23102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:39.717765 23102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:39.717780 23102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:39.719527 23102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:46359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:46359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:39.719761 23102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:39.719992 23102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:39.724740 23119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:39.724784 23122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:39.724906 23116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:39.724922 23102 server_base.cc:1034] running on GCE node
I20260812 14:42:39.725179 23102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:39.725433 23102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:39.726557 23102 hybrid_clock.cc:648] HybridClock initialized: now 1786545759726546 us; error 30 us; skew 500 ppm
Aug 12 14:42:39 dist-test-slave-q9h9 krb5kdc[23077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:39.728843 23102 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:39.729239 23102 webserver.cc:460] Webserver started at http://127.15.227.254:35411/ using document root <none> and password file <none>
I20260812 14:42:39.729468 23102 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:39.729516 23102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:39.729637 23102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:39.730614 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "cc445e8b05584707b21c18d0783ba48e"
format_stamp: "Formatted at 2026-08-12 14:42:39 on dist-test-slave-q9h9"
server_key: "40d60acf76461ad3d34a94613d32a2fb"
server_key_iv: "28e2d25f0c465911e42b380642b7917a"
server_key_version: "encryptionkey@0"
I20260812 14:42:39.730933 23102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "cc445e8b05584707b21c18d0783ba48e"
format_stamp: "Formatted at 2026-08-12 14:42:39 on dist-test-slave-q9h9"
server_key: "40d60acf76461ad3d34a94613d32a2fb"
server_key_iv: "28e2d25f0c465911e42b380642b7917a"
server_key_version: "encryptionkey@0"
I20260812 14:42:39.732151 23102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:39.732852 23136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:39.733019 23102 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:39.733100 23102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "cc445e8b05584707b21c18d0783ba48e"
format_stamp: "Formatted at 2026-08-12 14:42:39 on dist-test-slave-q9h9"
server_key: "40d60acf76461ad3d34a94613d32a2fb"
server_key_iv: "28e2d25f0c465911e42b380642b7917a"
server_key_version: "encryptionkey@0"
I20260812 14:42:39.733158 23102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:39.755201 23102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:39.757177 23102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:39.757313 23102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:39.761215 23102 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:46359
I20260812 14:42:39.761266 23234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:46359 every 8 connection(s)
I20260812 14:42:39.761718 23102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:39.762369 23236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:39.764523 23236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e: Bootstrap starting.
I20260812 14:42:39.765093 23236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:39.765360 23236 log.cc:824] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:39.765971 23236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e: No bootstrap required, opened a new log
I20260812 14:42:39.767493 23236 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc445e8b05584707b21c18d0783ba48e" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46359 } }
I20260812 14:42:39.767622 23236 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:39.767664 23236 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc445e8b05584707b21c18d0783ba48e, State: Initialized, Role: FOLLOWER
I20260812 14:42:39.767799 23236 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc445e8b05584707b21c18d0783ba48e" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46359 } }
I20260812 14:42:39.767858 23236 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:39.767881 23236 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:39.767925 23236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:39.768746 23236 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc445e8b05584707b21c18d0783ba48e" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46359 } }
I20260812 14:42:39.768836 23236 leader_election.cc:302] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc445e8b05584707b21c18d0783ba48e; no voters:
I20260812 14:42:39.768986 23236 leader_election.cc:288] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:39.769032 23239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:39.769207 23236 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:39.769219 23239 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [term 1 LEADER]: Becoming Leader. State: Replica: cc445e8b05584707b21c18d0783ba48e, State: Running, Role: LEADER
I20260812 14:42:39.769368 23239 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc445e8b05584707b21c18d0783ba48e" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46359 } }
I20260812 14:42:39.769726 23246 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc445e8b05584707b21c18d0783ba48e. Latest consensus state: current_term: 1 leader_uuid: "cc445e8b05584707b21c18d0783ba48e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc445e8b05584707b21c18d0783ba48e" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46359 } } }
I20260812 14:42:39.770076 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23102
I20260812 14:42:39.770191 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:39.770391 16271 external_mini_cluster.cc:1428] Setting key 6afc20e55c6c30f9f960be4b171888d1
I20260812 14:42:39.770583 23246 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:39.769737 23240 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc445e8b05584707b21c18d0783ba48e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc445e8b05584707b21c18d0783ba48e" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 46359 } } }
I20260812 14:42:39.770833 23240 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e [sys.catalog]: This master's current role is: LEADER
W20260812 14:42:39.771018 23269 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:42:39.771073 23269 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:42:39.771163 23276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:39.771345 23276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:39.772727 23276 catalog_manager.cc:1347] Generated new cluster ID: 975571434ed34b10b645db9f6eb3e5f7
I20260812 14:42:39.772773 23276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:42:39 dist-test-slave-q9h9 krb5kdc[23077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:39.781144 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:39.770944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.15.227.254:46359)
0812 14:42:39.771112 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:39.771121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:39.771165 (+ 44us) server_negotiation.cc:407] Connection header received
0812 14:42:39.771277 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:39.771284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:39.771338 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:39.771473 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:39.772050 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:39.773379 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:39.774426 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:39.775069 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:39.775191 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:39.777788 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:39.777792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:39.777792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:39.777803 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:39.779613 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:39.779996 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:39.779999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:39.779999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:39.780089 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:39.780292 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:39.780292 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:39.780293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:39.780567 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:39.780713 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:39.780812 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 14:42:39.780935 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
I20260812 14:42:39.793721 23276 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:39.794272 23276 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:39.796710 23276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc445e8b05584707b21c18d0783ba48e: Generated new TSK 0
I20260812 14:42:39.796866 23276 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:39.824918 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46359
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:39.903945 23286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:39.904129 23286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:39.904152 23286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:39.905248 23286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:39.905288 23286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:39.905313 23286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:39.906787 23286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:39.907002 23286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:39.907204 23286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:39.911917 23297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:39.911895 23294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:39.912034 23286 server_base.cc:1034] running on GCE node
W20260812 14:42:39.911962 23295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:39.912326 23286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:39.912540 23286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:39.913674 23286 hybrid_clock.cc:648] HybridClock initialized: now 1786545759913657 us; error 32 us; skew 500 ppm
Aug 12 14:42:39 dist-test-slave-q9h9 krb5kdc[23077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:39.916093 23286 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:39.916466 23286 webserver.cc:460] Webserver started at http://127.15.227.193:38153/ using document root <none> and password file <none>
I20260812 14:42:39.916656 23286 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:39.916694 23286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:39.916785 23286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:39.917690 23286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "2d8d97b2cdeb410686be68dc4f5fd720"
format_stamp: "Formatted at 2026-08-12 14:42:39 on dist-test-slave-q9h9"
server_key: "1e661f91d769dec1688bdec3560a11e4"
server_key_iv: "d2b046d2375401a2a8e4a65729f88d65"
server_key_version: "encryptionkey@0"
I20260812 14:42:39.917966 23286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "2d8d97b2cdeb410686be68dc4f5fd720"
format_stamp: "Formatted at 2026-08-12 14:42:39 on dist-test-slave-q9h9"
server_key: "1e661f91d769dec1688bdec3560a11e4"
server_key_iv: "d2b046d2375401a2a8e4a65729f88d65"
server_key_version: "encryptionkey@0"
I20260812 14:42:39.919209 23286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:39.919818 23311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:39.919955 23286 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:39.920037 23286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "2d8d97b2cdeb410686be68dc4f5fd720"
format_stamp: "Formatted at 2026-08-12 14:42:39 on dist-test-slave-q9h9"
server_key: "1e661f91d769dec1688bdec3560a11e4"
server_key_iv: "d2b046d2375401a2a8e4a65729f88d65"
server_key_version: "encryptionkey@0"
I20260812 14:42:39.920095 23286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:39.930095 23286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:39.932004 23286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:39.932132 23286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:39.932320 23286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:39.932590 23286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:39.932631 23286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:39.932657 23286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:39.932682 23286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:39.938293 23286 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:33931
I20260812 14:42:39.938349 23527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:33931 every 8 connection(s)
I20260812 14:42:39.938622 23286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:39.939414 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23286
I20260812 14:42:39.939525 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:39.939699 16271 external_mini_cluster.cc:1428] Setting key 344c35bbfd43f4eb42a1f4e97c203bce
Aug 12 14:42:39 dist-test-slave-q9h9 krb5kdc[23077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:39.947885 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:39.939219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:56045 (local address 127.15.227.254:46359)
0812 14:42:39.939299 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:39.939302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:39.939521 (+ 219us) server_negotiation.cc:407] Connection header received
0812 14:42:39.940636 (+ 1115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:39.940636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:39.940649 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:39.940686 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:39.941418 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:39.941896 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:39.942908 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:39.943112 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:39.943168 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:39.945263 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:39.945265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:39.945265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:39.945272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:39.946521 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:39.946967 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:39.946968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:39.946968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:39.947014 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:39.947270 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:39.947271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:39.947271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:39.947362 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:39.947406 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:39.947754 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 14:42:39.947822 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
I20260812 14:42:39.948089 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:39.939200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:46359 (local address 127.15.227.193:56045)
0812 14:42:39.939382 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:39.939428 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:39.940524 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:39.940754 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:39.940757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:39.941079 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:39.941336 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:39.941342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:39.941988 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:39.941989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:39.942820 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:39.942823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:39.943186 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:39.943187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:39.943336 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:39.943762 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:39.943770 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:39.945161 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:39.946607 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:39.946615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:39.946617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:39.946897 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:39.947066 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:39.947067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:39.947067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:39.947209 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:39.947419 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:39.947426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:39.947586 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 14:42:39.947716 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 14:42:39.947874 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260812 14:42:39.948688 23528 heartbeater.cc:347] Connected to a master server at 127.15.227.254:46359
I20260812 14:42:39.948772 23528 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:39.948949 23528 heartbeater.cc:511] Master 127.15.227.254:46359 requested a full tablet report, sending...
I20260812 14:42:39.949427 23168 ts_manager.cc:194] Registered new tserver with Master: 2d8d97b2cdeb410686be68dc4f5fd720 (127.15.227.193:33931)
I20260812 14:42:39.950110 23168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:56045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:39.962379 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46359
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:40.042802 23535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:40.042992 23535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:40.043016 23535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:40.044106 23535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:40.044145 23535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:40.044173 23535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:40.045609 23535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:40.045825 23535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:40.046047 23535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:40.050765 23544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:40.050871 23548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:40.051003 23535 server_base.cc:1034] running on GCE node
W20260812 14:42:40.050872 23546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:40.051231 23535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:40.051421 23535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:40.052558 23535 hybrid_clock.cc:648] HybridClock initialized: now 1786545760052547 us; error 27 us; skew 500 ppm
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[23077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545760, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:40.054940 23535 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:40.055356 23535 webserver.cc:460] Webserver started at http://127.15.227.194:35211/ using document root <none> and password file <none>
I20260812 14:42:40.055523 23535 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:40.055560 23535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:40.055650 23535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:40.056588 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "4020458a607a417387563409ecef47a5"
format_stamp: "Formatted at 2026-08-12 14:42:40 on dist-test-slave-q9h9"
server_key: "5e52e14b7d4a5314905f46013a877157"
server_key_iv: "023b6bd66c5706fea877e05c27fed4ac"
server_key_version: "encryptionkey@0"
I20260812 14:42:40.056874 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "4020458a607a417387563409ecef47a5"
format_stamp: "Formatted at 2026-08-12 14:42:40 on dist-test-slave-q9h9"
server_key: "5e52e14b7d4a5314905f46013a877157"
server_key_iv: "023b6bd66c5706fea877e05c27fed4ac"
server_key_version: "encryptionkey@0"
I20260812 14:42:40.058050 23535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:40.058715 23569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:40.058864 23535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:40.058944 23535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "4020458a607a417387563409ecef47a5"
format_stamp: "Formatted at 2026-08-12 14:42:40 on dist-test-slave-q9h9"
server_key: "5e52e14b7d4a5314905f46013a877157"
server_key_iv: "023b6bd66c5706fea877e05c27fed4ac"
server_key_version: "encryptionkey@0"
I20260812 14:42:40.058997 23535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:40.078486 23535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:40.080113 23535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:40.080224 23535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:40.080397 23535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:40.080632 23535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:40.080662 23535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:40.080679 23535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:40.080690 23535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:40.085845 23535 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:45205
I20260812 14:42:40.085911 23797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:45205 every 8 connection(s)
I20260812 14:42:40.086194 23535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:40.086628 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23535
I20260812 14:42:40.086716 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:40.086881 16271 external_mini_cluster.cc:1428] Setting key 7478cb615760793eba756c2b10ad5b7d
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[23077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545760, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:40.095603 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:40.086830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:33913 (local address 127.15.227.254:46359)
0812 14:42:40.086881 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:40.086884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:40.087094 (+ 210us) server_negotiation.cc:407] Connection header received
0812 14:42:40.088090 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:40.088091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:40.088103 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:40.088170 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:40.088706 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.089356 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.090668 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.090926 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.090994 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:40.092914 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:40.092915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:40.092916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:40.092925 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:40.094400 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.094744 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.094745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.094746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.094821 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.095024 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.095025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.095025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.095157 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:40.095231 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:40.095430 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 14:42:40.095517 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
I20260812 14:42:40.095642 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:40.086826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:46359 (local address 127.15.227.194:33913)
0812 14:42:40.086982 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:40.087019 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:40.087991 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:40.088223 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:40.088226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:40.088465 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:40.088633 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:40.088637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.089447 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.089448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:40.090586 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:40.090588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.090996 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.090997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:40.091087 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:40.091506 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:40.091514 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:40.092829 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:40.094470 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:40.094476 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:40.094477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:40.094676 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:40.094881 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:40.094882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:40.094882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:40.094968 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:40.095217 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:40.095220 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:40.095321 (+ 101us) client_negotiation.cc:769] Sending connection context
0812 14:42:40.095393 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:42:40.095498 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260812 14:42:40.096057 23798 heartbeater.cc:347] Connected to a master server at 127.15.227.254:46359
I20260812 14:42:40.096138 23798 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:40.096297 23798 heartbeater.cc:511] Master 127.15.227.254:46359 requested a full tablet report, sending...
I20260812 14:42:40.096674 23168 ts_manager.cc:194] Registered new tserver with Master: 4020458a607a417387563409ecef47a5 (127.15.227.194:45205)
I20260812 14:42:40.097028 23168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:33913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:40.110265 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46359
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:40.189548 23808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:40.189739 23808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:40.189769 23808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:40.190928 23808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:40.190969 23808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:40.191004 23808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:40.192469 23808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:46359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:40.192685 23808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:40.192885 23808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:40.197629 23825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:40.197635 23822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:40.197635 23821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:40.197808 23808 server_base.cc:1034] running on GCE node
I20260812 14:42:40.198065 23808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:40.198300 23808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:40.199450 23808 hybrid_clock.cc:648] HybridClock initialized: now 1786545760199439 us; error 26 us; skew 500 ppm
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[23077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545760, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:40.201813 23808 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:40.202278 23808 webserver.cc:460] Webserver started at http://127.15.227.195:34451/ using document root <none> and password file <none>
I20260812 14:42:40.202471 23808 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:40.202517 23808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:40.202641 23808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:40.203598 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "c50b8b6ae052479b9f02a208215945a3"
format_stamp: "Formatted at 2026-08-12 14:42:40 on dist-test-slave-q9h9"
server_key: "9c3f192d183ef47e7922608d2e74c9c2"
server_key_iv: "c320c6feccaa874caa2846a78b71983e"
server_key_version: "encryptionkey@0"
I20260812 14:42:40.203922 23808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "c50b8b6ae052479b9f02a208215945a3"
format_stamp: "Formatted at 2026-08-12 14:42:40 on dist-test-slave-q9h9"
server_key: "9c3f192d183ef47e7922608d2e74c9c2"
server_key_iv: "c320c6feccaa874caa2846a78b71983e"
server_key_version: "encryptionkey@0"
I20260812 14:42:40.205118 23808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:42:40.205754 23842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:40.205900 23808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:40.205981 23808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "c50b8b6ae052479b9f02a208215945a3"
format_stamp: "Formatted at 2026-08-12 14:42:40 on dist-test-slave-q9h9"
server_key: "9c3f192d183ef47e7922608d2e74c9c2"
server_key_iv: "c320c6feccaa874caa2846a78b71983e"
server_key_version: "encryptionkey@0"
I20260812 14:42:40.206071 23808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:40.220558 23808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:40.222159 23808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:40.222285 23808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:40.222471 23808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:40.222744 23808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:40.222779 23808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:40.222805 23808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:40.222827 23808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:40.228174 23808 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:46863
I20260812 14:42:40.228211 24074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:46863 every 8 connection(s)
I20260812 14:42:40.228483 23808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[23077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545760, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:40.234190 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23808
I20260812 14:42:40.234295 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:40.234494 16271 external_mini_cluster.cc:1428] Setting key b61533073214de5453084aa7045ee3e8
I20260812 14:42:40.237006 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:40.229125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:50207 (local address 127.15.227.254:46359)
0812 14:42:40.229200 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:40.229203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:40.229325 (+ 122us) server_negotiation.cc:407] Connection header received
0812 14:42:40.230202 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:40.230203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:40.230215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:40.230264 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:40.230784 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.231290 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.232204 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.232398 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.232468 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:40.234476 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:40.234478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:40.234479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:40.234486 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:40.235924 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.236271 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.236272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.236273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.236326 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.236534 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.236534 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.236535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.236633 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:40.236680 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:40.236888 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 14:42:40.236944 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 14:42:40.237154 24080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:40.229085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:46359 (local address 127.15.227.195:50207)
0812 14:42:40.229213 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:40.229255 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:40.230118 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:40.230316 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:40.230319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:40.230543 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:40.230717 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:40.230721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.231352 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.231352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:40.232104 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:40.232105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.232465 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.232466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:40.232553 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:40.232985 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:40.232994 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:40.234367 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:40.235992 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:40.235999 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:40.236000 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:40.236208 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:40.236376 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:40.236376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:40.236376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:40.236486 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:40.236680 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:40.236685 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:40.236793 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 14:42:40.236869 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:42:40.236980 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":67,"threads_started":1}
I20260812 14:42:40.237541 24078 heartbeater.cc:347] Connected to a master server at 127.15.227.254:46359
I20260812 14:42:40.237624 24078 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:40.237761 24078 heartbeater.cc:511] Master 127.15.227.254:46359 requested a full tablet report, sending...
I20260812 14:42:40.238107 23168 ts_manager.cc:194] Registered new tserver with Master: c50b8b6ae052479b9f02a208215945a3 (127.15.227.195:46863)
I20260812 14:42:40.238469 23168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:50207
I20260812 14:42:40.246196 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[23077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[23077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:40.268486 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:40.261551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33390 (local address 127.15.227.193:33931)
0812 14:42:40.261729 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:40.261733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:40.261743 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:40.261766 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:40.261768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:40.261781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:40.261850 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:40.262456 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.263159 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.263881 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.264165 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.264211 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:40.266269 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:40.266277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:40.266277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:40.266284 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:40.267556 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.267852 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.267855 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.267855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.267918 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.268041 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.268042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.268042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.268161 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:40.268255 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:40.268327 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:42:40.268387 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[23077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:40.275574 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:40.269654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.15.227.254:46359)
0812 14:42:40.269718 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:40.269720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:40.269730 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:40.269802 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:40.269803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:40.269812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:40.269853 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:40.270348 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.270938 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.271624 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:40.271823 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:40.271875 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:40.273675 (+ 1800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:40.273676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:40.273677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:40.273683 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:40.274767 (+ 1084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.275039 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.275040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.275040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.275083 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:40.275205 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:40.275205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:40.275206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:40.275325 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:40.275392 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:40.275457 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:42:40.275516 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260812 14:42:40.275777 23168 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58966
I20260812 14:42:40.276072 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23286
I20260812 14:42:40.284562 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23535
I20260812 14:42:40.307539 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23808
I20260812 14:42:40.314438 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23102
2026-08-12T14:42:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1973 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[24104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[24104](info): set up 2 sockets
Aug 12 14:42:40 dist-test-slave-q9h9 krb5kdc[24104](info): commencing operation
krb5kdc: starting...
W20260812 14:42:41.511165 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:41 dist-test-slave-q9h9 krb5kdc[24104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:41.622928 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43893
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:43893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:41.725124 24120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:41.725327 24120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:41.725358 24120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:41.726540 24120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:41.726580 24120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:41.726599 24120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:41.726614 24120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:41.726630 24120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:41.728199 24120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:43893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:41.728452 24120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:41.728658 24120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:41.733219 24129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:41.733201 24130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:41.733392 24120 server_base.cc:1034] running on GCE node
W20260812 14:42:41.733201 24135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:41.733664 24120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:41.733927 24120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:41.735059 24120 hybrid_clock.cc:648] HybridClock initialized: now 1786545761735051 us; error 44 us; skew 500 ppm
Aug 12 14:42:41 dist-test-slave-q9h9 krb5kdc[24104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545761, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:41.737382 24120 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:41.737679 24120 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:41.737725 24120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:41.737872 24120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:41.738929 24120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637"
format_stamp: "Formatted at 2026-08-12 14:42:41 on dist-test-slave-q9h9"
server_key: "c6f7c24c069532c2f4c89590de93a919"
server_key_iv: "57c49439fb6e9d7b7ac84231e841d6df"
server_key_version: "encryptionkey@0"
I20260812 14:42:41.739276 24120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637"
format_stamp: "Formatted at 2026-08-12 14:42:41 on dist-test-slave-q9h9"
server_key: "c6f7c24c069532c2f4c89590de93a919"
server_key_iv: "57c49439fb6e9d7b7ac84231e841d6df"
server_key_version: "encryptionkey@0"
I20260812 14:42:41.740465 24120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:41.741107 24145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:41.741266 24120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:41.741343 24120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637"
format_stamp: "Formatted at 2026-08-12 14:42:41 on dist-test-slave-q9h9"
server_key: "c6f7c24c069532c2f4c89590de93a919"
server_key_iv: "57c49439fb6e9d7b7ac84231e841d6df"
server_key_version: "encryptionkey@0"
I20260812 14:42:41.741408 24120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:41.755776 24120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:41.757736 24120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:41.757848 24120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:41.761714 24120 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:43893
I20260812 14:42:41.761758 24250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:43893 every 8 connection(s)
I20260812 14:42:41.762039 24120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:41.763103 24251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:41.765419 24251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637: Bootstrap starting.
I20260812 14:42:41.766004 24251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:41.766314 24251 log.cc:824] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:41.766937 24251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637: No bootstrap required, opened a new log
I20260812 14:42:41.768496 24251 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43893 } }
I20260812 14:42:41.768633 24251 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:41.768677 24251 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0a0a24ef9bc4fbfbbb8f069e6477637, State: Initialized, Role: FOLLOWER
I20260812 14:42:41.768815 24251 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43893 } }
I20260812 14:42:41.768894 24251 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:41.768934 24251 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:41.768982 24251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:41.769707 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24120
I20260812 14:42:41.769812 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:41.769892 24251 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43893 } }
I20260812 14:42:41.770004 16271 external_mini_cluster.cc:1428] Setting key ecdde8662cbf18e8dee2bfbaf4b98333
I20260812 14:42:41.769999 24251 leader_election.cc:302] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0a0a24ef9bc4fbfbbb8f069e6477637; no voters:
I20260812 14:42:41.770175 24251 leader_election.cc:288] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:41.770229 24255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:41.770387 24255 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [term 1 LEADER]: Becoming Leader. State: Replica: d0a0a24ef9bc4fbfbbb8f069e6477637, State: Running, Role: LEADER
I20260812 14:42:41.770481 24255 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43893 } }
I20260812 14:42:41.770516 24251 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:41.770787 24256 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43893 } } }
I20260812 14:42:41.770861 24256 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:41.771011 24261 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0a0a24ef9bc4fbfbbb8f069e6477637. Latest consensus state: current_term: 1 leader_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0a0a24ef9bc4fbfbbb8f069e6477637" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43893 } } }
I20260812 14:42:41.771059 24266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:41.771075 24261 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:41.771574 24266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:41.775053 24266 catalog_manager.cc:1347] Generated new cluster ID: 96df3864baa84ce3bd2a8d9ab71c7dc3
I20260812 14:42:41.775108 24266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:42:41 dist-test-slave-q9h9 krb5kdc[24104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:41.782548 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:41.770522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48270 (local address 127.15.227.254:43893)
0812 14:42:41.771141 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:41.771200 (+ 59us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:41.772096 (+ 896us) server_negotiation.cc:407] Connection header received
0812 14:42:41.772263 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:41.772272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:41.772337 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:41.772481 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:41.773217 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:41.774592 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:41.775671 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:41.776282 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:41.776391 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:41.779114 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:41.779117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:41.779118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:41.779129 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:41.780948 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:41.781347 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:41.781349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:41.781350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:41.781446 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:41.781638 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:41.781638 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:41.781639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:41.781908 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:41.782062 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:41.782183 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 14:42:41.782321 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":64,"threads_started":1}
I20260812 14:42:41.785274 24266 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:41.785794 24266 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:41.801234 24266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637: Generated new TSK 0
I20260812 14:42:41.801483 24266 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:41.833338 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43893
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:41.915779 24295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:41.915962 24295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:41.915984 24295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:41.917044 24295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:41.917075 24295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:41.917099 24295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:41.918606 24295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:41.918800 24295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:41.918967 24295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:41.923691 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:41.923712 24309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:41.923725 24307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:41.924053 24295 server_base.cc:1034] running on GCE node
I20260812 14:42:41.924279 24295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:41.924480 24295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:41.925655 24295 hybrid_clock.cc:648] HybridClock initialized: now 1786545761925638 us; error 32 us; skew 500 ppm
Aug 12 14:42:41 dist-test-slave-q9h9 krb5kdc[24104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545761, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:41.928093 24295 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:41.928385 24295 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:41.928428 24295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:41.928516 24295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:41.929437 24295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "d066e1cd67bc4860b3449b8e01b1c31a"
format_stamp: "Formatted at 2026-08-12 14:42:41 on dist-test-slave-q9h9"
server_key: "574ca12633fd7fe0e726fc8c01a7f1e1"
server_key_iv: "5cdb98140788226752b5927a0184e546"
server_key_version: "encryptionkey@0"
I20260812 14:42:41.929723 24295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "d066e1cd67bc4860b3449b8e01b1c31a"
format_stamp: "Formatted at 2026-08-12 14:42:41 on dist-test-slave-q9h9"
server_key: "574ca12633fd7fe0e726fc8c01a7f1e1"
server_key_iv: "5cdb98140788226752b5927a0184e546"
server_key_version: "encryptionkey@0"
I20260812 14:42:41.930950 24295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:42:41.931579 24318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:41.931730 24295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:41.931811 24295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "d066e1cd67bc4860b3449b8e01b1c31a"
format_stamp: "Formatted at 2026-08-12 14:42:41 on dist-test-slave-q9h9"
server_key: "574ca12633fd7fe0e726fc8c01a7f1e1"
server_key_iv: "5cdb98140788226752b5927a0184e546"
server_key_version: "encryptionkey@0"
I20260812 14:42:41.931870 24295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:41.942807 24295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:41.944795 24295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:41.944921 24295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:41.945123 24295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:41.945405 24295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:41.945436 24295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:41.945461 24295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:41.945487 24295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:41.950901 24295 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:39175
I20260812 14:42:41.950954 24532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:39175 every 8 connection(s)
I20260812 14:42:41.951170 24295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:42:41 dist-test-slave-q9h9 krb5kdc[24104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545761, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:41.958737 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24295
I20260812 14:42:41.958841 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:41.959012 16271 external_mini_cluster.cc:1428] Setting key 7d668b0c19d755cacd0cd6a62b8ddbcb
I20260812 14:42:41.959348 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:41.951799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59625 (local address 127.15.227.254:43893)
0812 14:42:41.951851 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:41.951852 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:41.952028 (+ 176us) server_negotiation.cc:407] Connection header received
0812 14:42:41.952481 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:41.952482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:41.952499 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:41.952550 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:41.953411 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:41.953907 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:41.954891 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:41.955103 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:41.955160 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:41.957049 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:41.957050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:41.957051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:41.957059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:41.958153 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:41.958485 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:41.958486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:41.958487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:41.958531 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:41.958727 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:41.958728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:41.958728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:41.958878 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:41.958941 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:41.959183 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 14:42:41.959261 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:42:41.959528 24540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:41.951769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43893 (local address 127.15.227.193:59625)
0812 14:42:41.951914 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:41.951949 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:41.952393 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:41.952616 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:41.952619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:41.952861 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:41.953321 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:41.953326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:41.953990 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:41.953991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:41.954812 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:41.954815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:41.955171 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:41.955172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:41.955284 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:41.955674 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:41.955681 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:41.956964 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:41.958217 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:41.958224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:41.958226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:41.958425 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:41.958572 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:41.958572 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:41.958572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:41.958662 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:41.958943 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:41.958948 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:41.959056 (+ 108us) client_negotiation.cc:769] Sending connection context
0812 14:42:41.959170 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 14:42:41.959324 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260812 14:42:41.960155 24536 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43893
I20260812 14:42:41.960253 24536 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:41.960397 24536 heartbeater.cc:511] Master 127.15.227.254:43893 requested a full tablet report, sending...
I20260812 14:42:41.960850 24174 ts_manager.cc:194] Registered new tserver with Master: d066e1cd67bc4860b3449b8e01b1c31a (127.15.227.193:39175)
I20260812 14:42:41.961433 24174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:59625
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:41.982010 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43893
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:42.063449 24545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:42.063642 24545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:42.063673 24545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:42.064790 24545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:42.064831 24545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:42.064864 24545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:42.066375 24545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:42.066596 24545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:42.066800 24545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:42.071319 24558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.071422 24552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.071458 24555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:42.071736 24545 server_base.cc:1034] running on GCE node
I20260812 14:42:42.071946 24545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:42.072188 24545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:42.073359 24545 hybrid_clock.cc:648] HybridClock initialized: now 1786545762073345 us; error 30 us; skew 500 ppm
Aug 12 14:42:42 dist-test-slave-q9h9 krb5kdc[24104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545762, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:42.075831 24545 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:42.076114 24545 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:42.076159 24545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:42.076268 24545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:42.077239 24545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "d613109d46294a5989a69c9c37d8863a"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "347e746d7e526c6b331cb6590c2fe1e5"
server_key_iv: "980bc8149ee4b2ff67bc47938ede83e2"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.077555 24545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "d613109d46294a5989a69c9c37d8863a"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "347e746d7e526c6b331cb6590c2fe1e5"
server_key_iv: "980bc8149ee4b2ff67bc47938ede83e2"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.078745 24545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:42:42.079391 24568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.079536 24545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:42.079617 24545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "d613109d46294a5989a69c9c37d8863a"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "347e746d7e526c6b331cb6590c2fe1e5"
server_key_iv: "980bc8149ee4b2ff67bc47938ede83e2"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.079674 24545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:42.089931 24545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:42.091861 24545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:42.091987 24545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:42.092180 24545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:42.092453 24545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:42.092484 24545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.092509 24545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:42.092527 24545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.097824 24545 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:33145
I20260812 14:42:42.097906 24790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:33145 every 8 connection(s)
I20260812 14:42:42.098150 24545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:42:42 dist-test-slave-q9h9 krb5kdc[24104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545762, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:42.106307 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.098770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:47159 (local address 127.15.227.254:43893)
0812 14:42:42.098834 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.098836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.099001 (+ 165us) server_negotiation.cc:407] Connection header received
0812 14:42:42.099424 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.099425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.099435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.099483 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:42.100312 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.100808 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.101693 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.101869 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.101912 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.103756 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:42.103758 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:42.103758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:42.103765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:42.105243 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:42.105579 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:42.105580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:42.105580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:42.105628 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:42.105794 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:42.105795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:42.105795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:42.105885 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:42.105929 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.106165 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.106232 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:42:42.106417 24793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.098724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43893 (local address 127.15.227.194:47159)
0812 14:42:42.098887 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.098926 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.099351 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.099539 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.099542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.099783 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:42.100247 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.100252 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.100867 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.100867 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.101630 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.101632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.101919 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.101920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.102005 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.102425 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:42.102432 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:42.103666 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:42.105322 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:42.105329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:42.105331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:42.105524 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:42.105670 (+ 146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:42.105671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:42.105671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:42.105754 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:42.105925 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:42.105927 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:42.106055 (+ 128us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.106141 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.106250 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260812 14:42:42.106639 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24545
I20260812 14:42:42.106729 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:42.106894 16271 external_mini_cluster.cc:1428] Setting key 1e545e475478464119369c732605cbcf
I20260812 14:42:42.107369 24791 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43893
I20260812 14:42:42.107448 24791 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:42.107583 24791 heartbeater.cc:511] Master 127.15.227.254:43893 requested a full tablet report, sending...
I20260812 14:42:42.107959 24171 ts_manager.cc:194] Registered new tserver with Master: d613109d46294a5989a69c9c37d8863a (127.15.227.194:33145)
I20260812 14:42:42.108311 24171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:47159
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:42.130795 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43893
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:42.211179 24802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:42.211366 24802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:42.211388 24802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:42.212471 24802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:42.212507 24802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:42.212530 24802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:42.213982 24802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:42.214198 24802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:42.214375 24802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:42.219002 24807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.219090 24816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.219105 24808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:42.219203 24802 server_base.cc:1034] running on GCE node
I20260812 14:42:42.219426 24802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:42.219611 24802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:42.220749 24802 hybrid_clock.cc:648] HybridClock initialized: now 1786545762220737 us; error 29 us; skew 500 ppm
Aug 12 14:42:42 dist-test-slave-q9h9 krb5kdc[24104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545762, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:42.223079 24802 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:42.223392 24802 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:42.223441 24802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:42.223564 24802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:42.224542 24802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "af059725e20d45deac4feca858d70062"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "b3e22de8d9e256d090c19dba6d4ee38f"
server_key_iv: "0e6fa401bf467eb077e2c94bfb77a92c"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.224867 24802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "af059725e20d45deac4feca858d70062"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "b3e22de8d9e256d090c19dba6d4ee38f"
server_key_iv: "0e6fa401bf467eb077e2c94bfb77a92c"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.226034 24802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:42.226729 24829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.226884 24802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:42.226965 24802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "af059725e20d45deac4feca858d70062"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "b3e22de8d9e256d090c19dba6d4ee38f"
server_key_iv: "0e6fa401bf467eb077e2c94bfb77a92c"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.227023 24802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:42.259593 24802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:42.261379 24802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:42.261512 24802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:42.261730 24802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:42.262082 24802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:42.262120 24802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.262151 24802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:42.262174 24802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.267932 24802 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:34545
I20260812 14:42:42.267974 25068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:34545 every 8 connection(s)
I20260812 14:42:42.268278 24802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:42 dist-test-slave-q9h9 krb5kdc[24104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545762, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:42.275646 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24802
I20260812 14:42:42.275758 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:42.275940 16271 external_mini_cluster.cc:1428] Setting key 99c807c2f3c87cfabaebb7904764c9a5
I20260812 14:42:42.277415 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.268950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:35401 (local address 127.15.227.254:43893)
0812 14:42:42.269012 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.269015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.269212 (+ 197us) server_negotiation.cc:407] Connection header received
0812 14:42:42.269696 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.269697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.269710 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.269763 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:42.270785 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.271387 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.272377 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.272573 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.272636 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.274701 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:42.274702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:42.274702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:42.274710 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:42.276131 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:42.276557 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:42.276559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:42.276559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:42.276621 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:42.276872 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:42.276873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:42.276874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:42.276982 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:42.277028 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.277277 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.277351 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:42:42.277500 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.268913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43893 (local address 127.15.227.195:35401)
0812 14:42:42.269076 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.269129 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.269594 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.269849 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.269852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.270170 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:42.270698 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.270704 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.271474 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.271475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.272289 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.272291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.272644 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.272645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.272760 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.273229 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:42.273238 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:42.274617 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:42.276206 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:42.276214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:42.276216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:42.276479 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:42.276674 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:42.276675 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:42.276676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:42.276809 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:42.277024 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:42.277028 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:42.277140 (+ 112us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.277217 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.277333 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
I20260812 14:42:42.277886 25070 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43893
I20260812 14:42:42.277971 25070 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:42.278136 25070 heartbeater.cc:511] Master 127.15.227.254:43893 requested a full tablet report, sending...
I20260812 14:42:42.278478 24171 ts_manager.cc:194] Registered new tserver with Master: af059725e20d45deac4feca858d70062 (127.15.227.195:34545)
I20260812 14:42:42.278817 24171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:35401
I20260812 14:42:42.287720 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:42:42.294540 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.288920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.15.227.254:43893)
0812 14:42:42.289005 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.289007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.289100 (+ 93us) server_negotiation.cc:407] Connection header received
0812 14:42:42.289172 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.289174 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.289185 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.289231 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:42.289769 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.290290 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.290972 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.291158 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.291212 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.292317 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:42.292318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:42.292319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:42.292326 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:42.293463 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:42.293776 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:42.293777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:42.293778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:42.293829 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:42.293989 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:42.293990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:42.293990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:42.294174 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:42.294266 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.294379 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.294454 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 14:42:42.296096 24171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:42.297641 24171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:42.304139 25095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.299768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:33145 (local address 127.0.0.1:34210)
0812 14:42:42.300142 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.300177 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.300232 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.300748 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.300749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.300751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:42.300800 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.300802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.301828 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.301829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.303191 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.303194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.303848 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.303849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.303928 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.303942 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.304012 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.304065 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":45,"thread_start_us":320,"threads_started":1}
I20260812 14:42:42.304159 25096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.300159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34210 (local address 127.15.227.194:33145)
0812 14:42:42.300567 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.300572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.300583 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:42.300620 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.300622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.300638 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.300713 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:42.300871 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.301628 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.303291 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.303772 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.303840 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.303897 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.304027 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.304065 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":84,"threads_started":1}
I20260812 14:42:42.304468 25087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.299770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:39175 (local address 127.0.0.1:43370)
0812 14:42:42.300148 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.300178 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.300244 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.300424 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.300428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.300430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:42.300493 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.300495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.301817 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.301818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.303193 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.303194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.304246 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.304247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.304318 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.304321 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.304371 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.304409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":48,"thread_start_us":68,"threads_started":1}
I20260812 14:42:42.304381 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.300349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.15.227.195:34545)
0812 14:42:42.300580 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.300585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.300921 (+ 336us) server_negotiation.cc:407] Connection header received
0812 14:42:42.300962 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.300964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.300979 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.301047 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:42.301169 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.302124 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.303146 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.303835 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.303903 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.303953 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.304143 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.304179 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
I20260812 14:42:42.304523 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.299884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.15.227.193:39175)
0812 14:42:42.300078 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.300083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.300247 (+ 164us) server_negotiation.cc:407] Connection header received
0812 14:42:42.300318 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.300321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.300337 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.300401 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:42.300597 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.301721 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.303273 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.303951 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.304020 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.304068 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.304401 (+ 333us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.304443 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260812 14:42:42.305099 24657 tablet_service.cc:1467] Processing CreateTablet for tablet bc0209b382ce4f9ba7d70ffe94d66381 (DEFAULT_TABLE table=test-table [id=2b71d1785776405dba3f2ee57ceb96c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:42.305398 24657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0209b382ce4f9ba7d70ffe94d66381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:42.306404 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.299983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:34545 (local address 127.0.0.1:53740)
0812 14:42:42.300812 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.300821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.300869 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.301057 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.301058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.301060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:42.301096 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.301097 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.302209 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.302210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.303071 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.303073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.303990 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.303991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.304057 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.304071 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.304116 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.304158 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"thread_start_us":50,"threads_started":1}
I20260812 14:42:42.306804 24401 tablet_service.cc:1467] Processing CreateTablet for tablet bc0209b382ce4f9ba7d70ffe94d66381 (DEFAULT_TABLE table=test-table [id=2b71d1785776405dba3f2ee57ceb96c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:42.307168 24401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0209b382ce4f9ba7d70ffe94d66381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:42.308522 25098 tablet_bootstrap.cc:492] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Bootstrap starting.
I20260812 14:42:42.309171 25098 tablet_bootstrap.cc:654] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:42.309638 25098 log.cc:824] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:42.310086 25101 tablet_bootstrap.cc:492] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a: Bootstrap starting.
I20260812 14:42:42.310405 25098 tablet_bootstrap.cc:492] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: No bootstrap required, opened a new log
I20260812 14:42:42.310474 25098 ts_tablet_manager.cc:1397] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:42.310690 25101 tablet_bootstrap.cc:654] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:42.310834 24943 tablet_service.cc:1467] Processing CreateTablet for tablet bc0209b382ce4f9ba7d70ffe94d66381 (DEFAULT_TABLE table=test-table [id=2b71d1785776405dba3f2ee57ceb96c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:42.311167 24943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc0209b382ce4f9ba7d70ffe94d66381. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:42.311249 25101 log.cc:824] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:42.311923 25098 raft_consensus.cc:348] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.312018 25101 tablet_bootstrap.cc:492] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a: No bootstrap required, opened a new log
I20260812 14:42:42.312036 25098 raft_consensus.cc:374] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:42.312057 25098 raft_consensus.cc:729] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d613109d46294a5989a69c9c37d8863a, State: Initialized, Role: FOLLOWER
I20260812 14:42:42.312073 25101 ts_tablet_manager.cc:1397] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:42.312152 25098 consensus_queue.cc:260] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.312407 25098 ts_tablet_manager.cc:1428] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:42:42.312417 24791 heartbeater.cc:503] Master 127.15.227.254:43893 was elected leader, sending a full tablet report...
I20260812 14:42:42.313948 25101 raft_consensus.cc:348] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.314164 25101 raft_consensus.cc:374] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:42.314213 25101 raft_consensus.cc:729] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d066e1cd67bc4860b3449b8e01b1c31a, State: Initialized, Role: FOLLOWER
I20260812 14:42:42.314320 25101 consensus_queue.cc:260] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.314399 25106 tablet_bootstrap.cc:492] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Bootstrap starting.
I20260812 14:42:42.314584 25101 ts_tablet_manager.cc:1428] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:42:42.314615 24536 heartbeater.cc:503] Master 127.15.227.254:43893 was elected leader, sending a full tablet report...
I20260812 14:42:42.315001 25106 tablet_bootstrap.cc:654] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:42.315547 25106 log.cc:824] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:42.316143 25106 tablet_bootstrap.cc:492] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: No bootstrap required, opened a new log
I20260812 14:42:42.316193 25106 ts_tablet_manager.cc:1397] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:42.317399 25106 raft_consensus.cc:348] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.317513 25106 raft_consensus.cc:374] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:42.317538 25106 raft_consensus.cc:729] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af059725e20d45deac4feca858d70062, State: Initialized, Role: FOLLOWER
I20260812 14:42:42.317624 25106 consensus_queue.cc:260] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.317881 25106 ts_tablet_manager.cc:1428] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:42:42.317889 25070 heartbeater.cc:503] Master 127.15.227.254:43893 was elected leader, sending a full tablet report...
W20260812 14:42:42.348593 24792 tablet.cc:2369] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:42:42.451824 24539 tablet.cc:2369] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:42.479321 25109 raft_consensus.cc:482] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:42.479444 25109 raft_consensus.cc:504] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.479812 25109 leader_election.cc:288] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d066e1cd67bc4860b3449b8e01b1c31a (127.15.227.193:39175), d613109d46294a5989a69c9c37d8863a (127.15.227.194:33145)
I20260812 14:42:42.482805 25088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.480037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:36897 (local address 127.15.227.193:39175)
0812 14:42:42.480082 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.480084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.480095 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:42.480197 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.480198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.480207 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.480253 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:42.480384 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.480849 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.481954 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.482500 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.482571 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.482579 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.482711 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.482748 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:42:42.482817 25075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.479994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:39175 (local address 127.15.227.195:36897)
0812 14:42:42.480049 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.480061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.480107 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.480262 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.480263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.480264 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:42.480330 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.480331 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.480917 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.480917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.481883 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.481886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.482577 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.482578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.482646 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.482649 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.482697 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.482735 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":18}
I20260812 14:42:42.482932 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.479993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:33145 (local address 127.15.227.195:34633)
0812 14:42:42.480176 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.480185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.480261 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.480387 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.480388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.480389 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:42.480456 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.480458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.481035 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.481035 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.482008 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:42.482011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.482726 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.482727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:42.482784 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.482788 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.482837 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.482869 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":13,"thread_start_us":94,"threads_started":1}
I20260812 14:42:42.482947 25096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.480086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:34633 (local address 127.15.227.194:33145)
0812 14:42:42.480126 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.480128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.480243 (+ 115us) server_negotiation.cc:407] Connection header received
0812 14:42:42.480321 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.480322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.480331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.480380 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:42.480508 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.480965 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.482101 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.482653 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.482715 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.482722 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.482849 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.482888 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:42:42.483227 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0209b382ce4f9ba7d70ffe94d66381" candidate_uuid: "af059725e20d45deac4feca858d70062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d613109d46294a5989a69c9c37d8863a" is_pre_election: true
I20260812 14:42:42.483361 24695 raft_consensus.cc:2454] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af059725e20d45deac4feca858d70062 in term 0.
I20260812 14:42:42.483366 24443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0209b382ce4f9ba7d70ffe94d66381" candidate_uuid: "af059725e20d45deac4feca858d70062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" is_pre_election: true
I20260812 14:42:42.483520 24443 raft_consensus.cc:2454] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af059725e20d45deac4feca858d70062 in term 0.
I20260812 14:42:42.483572 24837 leader_election.cc:302] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af059725e20d45deac4feca858d70062, d613109d46294a5989a69c9c37d8863a; no voters:
I20260812 14:42:42.483713 25109 raft_consensus.cc:2781] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:42.483758 25109 raft_consensus.cc:482] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:42.483780 25109 raft_consensus.cc:3037] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:42.484462 25109 raft_consensus.cc:504] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.484578 25109 leader_election.cc:288] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [CANDIDATE]: Term 1 election: Requested vote from peers d066e1cd67bc4860b3449b8e01b1c31a (127.15.227.193:39175), d613109d46294a5989a69c9c37d8863a (127.15.227.194:33145)
I20260812 14:42:42.484723 24695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0209b382ce4f9ba7d70ffe94d66381" candidate_uuid: "af059725e20d45deac4feca858d70062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d613109d46294a5989a69c9c37d8863a"
I20260812 14:42:42.484799 24695 raft_consensus.cc:3037] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:42.485464 24695 raft_consensus.cc:2454] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af059725e20d45deac4feca858d70062 in term 1.
I20260812 14:42:42.485582 24443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc0209b382ce4f9ba7d70ffe94d66381" candidate_uuid: "af059725e20d45deac4feca858d70062" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d066e1cd67bc4860b3449b8e01b1c31a"
I20260812 14:42:42.485630 24837 leader_election.cc:302] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af059725e20d45deac4feca858d70062, d613109d46294a5989a69c9c37d8863a; no voters:
I20260812 14:42:42.485673 24443 raft_consensus.cc:3037] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:42.485718 25109 raft_consensus.cc:2781] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:42.485854 25109 raft_consensus.cc:686] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 1 LEADER]: Becoming Leader. State: Replica: af059725e20d45deac4feca858d70062, State: Running, Role: LEADER
I20260812 14:42:42.485939 25109 consensus_queue.cc:237] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } }
I20260812 14:42:42.486488 24443 raft_consensus.cc:2454] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af059725e20d45deac4feca858d70062 in term 1.
I20260812 14:42:42.486791 24171 catalog_manager.cc:5495] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 reported cstate change: term changed from 0 to 1, leader changed from <none> to af059725e20d45deac4feca858d70062 (127.15.227.195). New cstate: current_term: 1 leader_uuid: "af059725e20d45deac4feca858d70062" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af059725e20d45deac4feca858d70062" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 34545 } health_report { overall_health: HEALTHY } } }
W20260812 14:42:42.518771 25074 tablet.cc:2369] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:42.530366 25097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.528125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.15.227.195:34545)
0812 14:42:42.528196 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.528197 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.528206 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:42.528239 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.528240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.528258 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.528319 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:42.528443 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.528897 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.529613 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:42.529814 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:42.529867 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:42.530008 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:42.530140 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:42.530219 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.530259 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.530279 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:42:42.531952 24443 raft_consensus.cc:1261] T bc0209b382ce4f9ba7d70ffe94d66381 P d066e1cd67bc4860b3449b8e01b1c31a [term 1 FOLLOWER]: Refusing update from remote peer af059725e20d45deac4feca858d70062: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:42.531952 24695 raft_consensus.cc:1261] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 1 FOLLOWER]: Refusing update from remote peer af059725e20d45deac4feca858d70062: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:42.532308 25111 consensus_queue.cc:1035] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d613109d46294a5989a69c9c37d8863a" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:42.532420 25109 consensus_queue.cc:1035] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d066e1cd67bc4860b3449b8e01b1c31a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:42.534873 25120 mvcc.cc:205] Tried to move back new op lower bound from 7317691443328479232 to 7317691443143094272. Current Snapshot: MvccSnapshot[applied={T|T < 7317691443328479232}]
I20260812 14:42:42.535545 25116 mvcc.cc:205] Tried to move back new op lower bound from 7317691443328479232 to 7317691443143094272. Current Snapshot: MvccSnapshot[applied={T|T < 7317691443328479232}]
I20260812 14:42:42.536720 24171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:42.536815 24171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:42.537485 24171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0a0a24ef9bc4fbfbbb8f069e6477637: Sending DeleteTablet for 3 replicas of tablet bc0209b382ce4f9ba7d70ffe94d66381
I20260812 14:42:42.537815 24401 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0209b382ce4f9ba7d70ffe94d66381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:42 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:43370
I20260812 14:42:42.537878 24943 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0209b382ce4f9ba7d70ffe94d66381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:42 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:53740
I20260812 14:42:42.538069 24657 tablet_service.cc:1514] Processing DeleteTablet for tablet bc0209b382ce4f9ba7d70ffe94d66381 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:42 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:34210
I20260812 14:42:42.538105 25135 tablet_replica.cc:330] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: stopping tablet replica
I20260812 14:42:42.538189 25135 raft_consensus.cc:2229] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:42.538241 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24295
I20260812 14:42:42.553292 25138 tablet_replica.cc:330] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: stopping tablet replica
I20260812 14:42:42.553478 25138 raft_consensus.cc:2229] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:42.553654 25138 raft_consensus.cc:2258] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:42.559166 25138 ts_tablet_manager.cc:1905] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:42.560817 25138 ts_tablet_manager.cc:1918] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:42.560956 25138 log.cc:1196] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/bc0209b382ce4f9ba7d70ffe94d66381
I20260812 14:42:42.561303 25138 ts_tablet_manager.cc:1939] T bc0209b382ce4f9ba7d70ffe94d66381 P d613109d46294a5989a69c9c37d8863a: Deleting consensus metadata
I20260812 14:42:42.561784 24151 catalog_manager.cc:4841] TS d613109d46294a5989a69c9c37d8863a (127.15.227.194:33145): tablet bc0209b382ce4f9ba7d70ffe94d66381 (table test-table [id=2b71d1785776405dba3f2ee57ceb96c6]) successfully deleted
I20260812 14:42:42.538308 25135 raft_consensus.cc:2258] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:42.564324 25135 ts_tablet_manager.cc:1905] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:42.565814 25135 ts_tablet_manager.cc:1918] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:42.565919 25135 log.cc:1196] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/bc0209b382ce4f9ba7d70ffe94d66381
I20260812 14:42:42.566202 25135 ts_tablet_manager.cc:1939] T bc0209b382ce4f9ba7d70ffe94d66381 P af059725e20d45deac4feca858d70062: Deleting consensus metadata
I20260812 14:42:42.566612 24154 catalog_manager.cc:4841] TS af059725e20d45deac4feca858d70062 (127.15.227.195:34545): tablet bc0209b382ce4f9ba7d70ffe94d66381 (table test-table [id=2b71d1785776405dba3f2ee57ceb96c6]) successfully deleted
W20260812 14:42:42.569271 24154 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:39175 (error 108)
I20260812 14:42:42.569589 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24545
I20260812 14:42:42.571393 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.569651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:39175 (local address 127.0.0.1:43372)
0812 14:42:42.571314 (+ 1663us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.571358 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:39175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1637}
W20260812 14:42:42.576592 24154 catalog_manager.cc:4584] TS d066e1cd67bc4860b3449b8e01b1c31a (127.15.227.193:39175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc0209b382ce4f9ba7d70ffe94d66381: Network error: Client connection negotiation failed: client connection to 127.15.227.193:39175: connect: Connection refused (error 111)
I20260812 14:42:42.605818 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24802
I20260812 14:42:42.614880 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24120
2026-08-12T14:42:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2300 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T14:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:42Z Disabled control of system clock
I20260812 14:42:42.646313 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36899
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:36899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:42:42.725059 25147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:42.725250 25147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:42.725270 25147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:42.726395 25147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:42.726432 25147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:42.726444 25147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:42.726454 25147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:42.726465 25147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:42.727777 25147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:36899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:42.728001 25147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:42.728266 25147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:42.732647 25158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.732671 25161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.732687 25157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:42.732849 25147 server_base.cc:1034] running on GCE node
I20260812 14:42:42.733112 25147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:42.733385 25147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:42.734517 25147 hybrid_clock.cc:648] HybridClock initialized: now 1786545762734500 us; error 31 us; skew 500 ppm
I20260812 14:42:42.735653 25147 webserver.cc:460] Webserver started at http://127.15.227.254:45793/ using document root <none> and password file <none>
I20260812 14:42:42.735862 25147 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:42.735911 25147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:42.736035 25147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:42.737076 25147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "4936ae361570460ab9894fad3abe34ff"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "f64bc644de42e3e38701a1ccce0f9e64"
server_key_iv: "07d7ea865cdec3876b47bbf04b14e3dc"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.737396 25147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "4936ae361570460ab9894fad3abe34ff"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "f64bc644de42e3e38701a1ccce0f9e64"
server_key_iv: "07d7ea865cdec3876b47bbf04b14e3dc"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.738646 25147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:42.739367 25174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.739499 25147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:42.739583 25147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "4936ae361570460ab9894fad3abe34ff"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "f64bc644de42e3e38701a1ccce0f9e64"
server_key_iv: "07d7ea865cdec3876b47bbf04b14e3dc"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.739643 25147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:42.742787 25147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:42.743022 25147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:42.743137 25147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:42.746794 25147 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:36899
I20260812 14:42:42.746827 25268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:36899 every 8 connection(s)
I20260812 14:42:42.747107 25147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:42.747713 25269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:42.749962 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25147
I20260812 14:42:42.750088 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:42.750247 16271 external_mini_cluster.cc:1428] Setting key dc61ec6ef468c9c9ad2b8be6e425b44e
I20260812 14:42:42.750455 25269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff: Bootstrap starting.
I20260812 14:42:42.751129 25269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:42.751495 25269 log.cc:824] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:42.752158 25269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff: No bootstrap required, opened a new log
I20260812 14:42:42.752719 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.750691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54040 (local address 127.15.227.254:36899)
0812 14:42:42.750899 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.750907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.750939 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:42:42.751482 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.751490 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.751542 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.751707 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:42.752313 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:42.752316 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:42.752317 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:42.752321 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:42.752368 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:42.752381 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:42.752463 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.752491 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.752545 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260812 14:42:42.753775 25269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4936ae361570460ab9894fad3abe34ff" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36899 } }
I20260812 14:42:42.753893 25269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:42.753913 25269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4936ae361570460ab9894fad3abe34ff, State: Initialized, Role: FOLLOWER
I20260812 14:42:42.753995 25269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4936ae361570460ab9894fad3abe34ff" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36899 } }
I20260812 14:42:42.754405 25269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:42.754442 25269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:42.754475 25269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:42.754936 25269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4936ae361570460ab9894fad3abe34ff" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36899 } }
I20260812 14:42:42.755018 25269 leader_election.cc:302] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4936ae361570460ab9894fad3abe34ff; no voters:
I20260812 14:42:42.755137 25269 leader_election.cc:288] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:42.755210 25276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:42.755349 25276 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [term 1 LEADER]: Becoming Leader. State: Replica: 4936ae361570460ab9894fad3abe34ff, State: Running, Role: LEADER
I20260812 14:42:42.755388 25269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:42.755505 25276 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4936ae361570460ab9894fad3abe34ff" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36899 } }
I20260812 14:42:42.755842 25278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4936ae361570460ab9894fad3abe34ff. Latest consensus state: current_term: 1 leader_uuid: "4936ae361570460ab9894fad3abe34ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4936ae361570460ab9894fad3abe34ff" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36899 } } }
I20260812 14:42:42.755947 25278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:42.755837 25277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4936ae361570460ab9894fad3abe34ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4936ae361570460ab9894fad3abe34ff" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36899 } } }
I20260812 14:42:42.756155 25286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:42.756233 25277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:42.756745 25286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:42.758112 25286 catalog_manager.cc:1347] Generated new cluster ID: a5409c84cfef44e4a0e2724412eceab1
I20260812 14:42:42.758163 25286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:42.790074 25286 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:42.790241 25286 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:42.798995 25286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff: Generated new TSK 0
I20260812 14:42:42.799182 25286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:42:42.800733 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36899
--builtin_ntp_servers=127.15.227.212:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:42:42.881932 25314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:42.882151 25314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:42.882184 25314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:42.883383 25314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:42.883423 25314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:42.883456 25314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:42.884763 25314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:42.884984 25314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:42.885252 25314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:42.889876 25325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.889885 25324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:42.889937 25327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:42.890183 25314 server_base.cc:1034] running on GCE node
I20260812 14:42:42.890360 25314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:42.890575 25314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:42.891724 25314 hybrid_clock.cc:648] HybridClock initialized: now 1786545762891709 us; error 31 us; skew 500 ppm
I20260812 14:42:42.892892 25314 webserver.cc:460] Webserver started at http://127.15.227.193:42167/ using document root <none> and password file <none>
I20260812 14:42:42.893092 25314 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:42.893136 25314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:42.893286 25314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:42.894336 25314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "1e5384fa956f49e99cb8d823d712698e"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "64d1661c0ec44e7df63be49289135574"
server_key_iv: "6595d7d78b4d5a1eb997ea0231cf4ae6"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.894654 25314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "1e5384fa956f49e99cb8d823d712698e"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "64d1661c0ec44e7df63be49289135574"
server_key_iv: "6595d7d78b4d5a1eb997ea0231cf4ae6"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.895861 25314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:42.896494 25337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.896620 25314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:42.896704 25314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "1e5384fa956f49e99cb8d823d712698e"
format_stamp: "Formatted at 2026-08-12 14:42:42 on dist-test-slave-q9h9"
server_key: "64d1661c0ec44e7df63be49289135574"
server_key_iv: "6595d7d78b4d5a1eb997ea0231cf4ae6"
server_key_version: "encryptionkey@0"
I20260812 14:42:42.896763 25314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:42.899950 25314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:42.900182 25314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:42.900297 25314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:42.900472 25314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:42.900749 25314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:42.900782 25314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.900808 25314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:42.900831 25314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:42.906222 25314 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:43423
I20260812 14:42:42.906257 25540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:43423 every 8 connection(s)
I20260812 14:42:42.906533 25314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:42.909092 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.907165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:60947 (local address 127.15.227.254:36899)
0812 14:42:42.907218 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:42.907219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:42.907412 (+ 193us) server_negotiation.cc:407] Connection header received
0812 14:42:42.907819 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:42.907820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:42.907825 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:42.907859 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:42.908939 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:42.908940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:42.908940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:42.908942 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:42.908951 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:42.908954 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:42.908984 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:42.909032 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:42:42.909042 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:42:42.909272 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:42.907139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36899 (local address 127.15.227.193:60947)
0812 14:42:42.907308 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:42.907345 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:42.907733 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:42.907914 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:42.907917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:42.907924 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:42.908826 (+ 902us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:42.908827 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:42.908853 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:42.908879 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:42.908989 (+ 110us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:42.908991 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:42.908993 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:42:42.909039 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:42:42.909082 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260812 14:42:42.909742 25542 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36899
I20260812 14:42:42.909816 25542 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:42.909941 25542 heartbeater.cc:511] Master 127.15.227.254:36899 requested a full tablet report, sending...
I20260812 14:42:42.910410 25202 ts_manager.cc:194] Registered new tserver with Master: 1e5384fa956f49e99cb8d823d712698e (127.15.227.193:43423)
I20260812 14:42:42.915918 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25314
I20260812 14:42:42.915993 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:42.916164 16271 external_mini_cluster.cc:1428] Setting key 4efb4c3624ee6457dc11ceb8a3397f5e
I20260812 14:42:42.916916 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36899
--builtin_ntp_servers=127.15.227.212:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 14:42:42.996599 25546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:42.996797 25546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:42.996819 25546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:42.997931 25546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:42.997970 25546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:42.997995 25546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:42.999366 25546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:42.999590 25546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:42.999863 25546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:43.004562 25560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:43.004696 25546 server_base.cc:1034] running on GCE node
W20260812 14:42:43.004577 25558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:43.004577 25562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:43.004989 25546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:43.005229 25546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:43.006371 25546 hybrid_clock.cc:648] HybridClock initialized: now 1786545763006356 us; error 28 us; skew 500 ppm
I20260812 14:42:43.013448 25546 webserver.cc:460] Webserver started at http://127.15.227.194:41411/ using document root <none> and password file <none>
I20260812 14:42:43.013701 25546 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:43.013764 25546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:43.013907 25546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:43.015090 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "fa42ca15edbe48ac9d425d523d9382ce"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "b76b59669a6925655abce8d8dff76227"
server_key_iv: "860191ce9a94a225eba88150f8041ceb"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.015473 25546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "fa42ca15edbe48ac9d425d523d9382ce"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "b76b59669a6925655abce8d8dff76227"
server_key_iv: "860191ce9a94a225eba88150f8041ceb"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.017006 25546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:42:43.017951 25572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.018256 25546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:43.018358 25546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "fa42ca15edbe48ac9d425d523d9382ce"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "b76b59669a6925655abce8d8dff76227"
server_key_iv: "860191ce9a94a225eba88150f8041ceb"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.018435 25546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:43.022291 25546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:43.022691 25546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:43.022838 25546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:43.023056 25546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:43.023408 25546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:43.023448 25546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.023476 25546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:43.023494 25546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.029098 25546 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:39201
I20260812 14:42:43.029191 25773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:39201 every 8 connection(s)
I20260812 14:42:43.029479 25546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:43.031807 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25546
I20260812 14:42:43.031917 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:43.032116 16271 external_mini_cluster.cc:1428] Setting key 9d41734cb0430f4f7096c2f2f5dd480d
I20260812 14:42:43.033077 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36899
--builtin_ntp_servers=127.15.227.212:37229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 14:42:43.033079 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.030261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:52563 (local address 127.15.227.254:36899)
0812 14:42:43.030317 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.030319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.030555 (+ 236us) server_negotiation.cc:407] Connection header received
0812 14:42:43.031232 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.031233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.031241 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.031294 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.032836 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.032838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.032838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.032843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.032859 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.032863 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.032908 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.032986 (+ 78us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.033000 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:42:43.033272 25777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.030236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36899 (local address 127.15.227.194:52563)
0812 14:42:43.030417 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.030459 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.031095 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.031368 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.031373 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.031382 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:43.032685 (+ 1303us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:43.032686 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:43.032716 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:43.032745 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:43.032914 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:43.032919 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:43.032922 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:43.032998 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:42:43.033062 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260812 14:42:43.033746 25775 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36899
I20260812 14:42:43.033849 25775 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:43.033998 25775 heartbeater.cc:511] Master 127.15.227.254:36899 requested a full tablet report, sending...
I20260812 14:42:43.034379 25202 ts_manager.cc:194] Registered new tserver with Master: fa42ca15edbe48ac9d425d523d9382ce (127.15.227.194:39201)
W20260812 14:42:43.112639 25780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:43.112815 25780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:43.112835 25780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:43.113937 25780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:43.113974 25780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:43.113997 25780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:43.115334 25780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:43.115530 25780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:43.115787 25780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:43.120396 25790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:43.120383 25798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:43.120424 25780 server_base.cc:1034] running on GCE node
W20260812 14:42:43.120369 25791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:43.120831 25780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:43.121060 25780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:43.122195 25780 hybrid_clock.cc:648] HybridClock initialized: now 1786545763122160 us; error 46 us; skew 500 ppm
I20260812 14:42:43.123390 25780 webserver.cc:460] Webserver started at http://127.15.227.195:42921/ using document root <none> and password file <none>
I20260812 14:42:43.123611 25780 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:43.123669 25780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:43.123799 25780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:43.124790 25780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "cfaeacf6e49d4bdd837c666a49adfc8f"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "ae46607fec654da66db217742a202c95"
server_key_iv: "d678294acd5ca10c77b18de6bd864b01"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.125113 25780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "cfaeacf6e49d4bdd837c666a49adfc8f"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "ae46607fec654da66db217742a202c95"
server_key_iv: "d678294acd5ca10c77b18de6bd864b01"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.126315 25780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:42:43.126967 25810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.127115 25780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:43.127197 25780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "cfaeacf6e49d4bdd837c666a49adfc8f"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "ae46607fec654da66db217742a202c95"
server_key_iv: "d678294acd5ca10c77b18de6bd864b01"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.127262 25780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:43.130424 25780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:43.130654 25780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:43.130766 25780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:43.130944 25780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:43.131209 25780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:43.131242 25780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.131266 25780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:43.131289 25780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.136660 25780 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:38219
I20260812 14:42:43.136718 26048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:38219 every 8 connection(s)
I20260812 14:42:43.136978 25780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:42:43.138200 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25780
I20260812 14:42:43.138306 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:43.138490 16271 external_mini_cluster.cc:1428] Setting key 846c4a55c64f678c47983d5e000a06bf
I20260812 14:42:43.139951 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.137629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:54555 (local address 127.15.227.254:36899)
0812 14:42:43.137700 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.137702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.137878 (+ 176us) server_negotiation.cc:407] Connection header received
0812 14:42:43.138513 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.138514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.138520 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.138571 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.139683 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.139684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.139684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.139687 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.139717 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.139721 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.139778 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.139884 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.139899 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:42:43.140117 26053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.137574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36899 (local address 127.15.227.195:54555)
0812 14:42:43.137774 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.137807 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.138399 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.138638 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.138642 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.138652 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:43.139557 (+ 905us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:43.139558 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:43.139583 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:43.139609 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:43.139805 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:43.139811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:43.139814 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:43.139886 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:42:43.139940 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260812 14:42:43.140547 26049 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36899
I20260812 14:42:43.140621 26049 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:43.140769 26049 heartbeater.cc:511] Master 127.15.227.254:36899 requested a full tablet report, sending...
I20260812 14:42:43.141134 25202 ts_manager.cc:194] Registered new tserver with Master: cfaeacf6e49d4bdd837c666a49adfc8f (127.15.227.195:38219)
I20260812 14:42:43.150043 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:42:43.152020 25273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.151221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54046 (local address 127.15.227.254:36899)
0812 14:42:43.151264 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.151265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.151378 (+ 113us) server_negotiation.cc:407] Connection header received
0812 14:42:43.151414 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.151414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.151420 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.151462 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.151874 (+ 412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.151875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.151875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.151878 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.151889 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.151893 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.151935 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.151963 (+ 28us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.151973 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:42:43.153290 25202 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54046:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:43.154840 25202 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:43.158192 26066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.156669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:43423 (local address 127.0.0.1:58620)
0812 14:42:43.156939 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.156976 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.157031 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.157277 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.157278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.157281 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:43.157743 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:43.157743 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:43.157757 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:43.157779 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:43.158048 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:43.158055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:43.158057 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:42:43.158115 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:42:43.158128 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"spinlock_wait_cycles":384,"thread_start_us":83,"threads_started":1}
I20260812 14:42:43.158277 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.156860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51912 (local address 127.15.227.195:38219)
0812 14:42:43.157101 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.157104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.157113 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:43.157133 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.157135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.157143 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.157204 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.157984 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.157991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.157991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.157995 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.158047 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.158067 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.158119 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.158184 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.158200 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":71,"threads_started":1}
I20260812 14:42:43.158440 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.156669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:38219 (local address 127.0.0.1:51912)
0812 14:42:43.157012 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.157020 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.157055 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.157232 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.157238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.157244 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:43.157909 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:43.157909 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:43.157916 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:43.157928 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:43.158126 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:43.158127 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:43.158128 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:42:43.158368 (+ 240us) client_negotiation.cc:240] Negotiation successful
0812 14:42:43.158382 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":4,"thread_start_us":55,"threads_started":1}
I20260812 14:42:43.158458 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.156766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.15.227.193:43423)
0812 14:42:43.157155 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.157159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.157168 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:43.157190 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.157192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.157201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.157269 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.157865 (+ 596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.157875 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.157875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.157879 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.157920 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.157947 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.158001 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.158361 (+ 360us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.158378 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":75,"threads_started":1}
I20260812 14:42:43.158797 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.156689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:39201 (local address 127.0.0.1:60554)
0812 14:42:43.157575 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.157585 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.157635 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.158052 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.158053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.158056 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:43.158466 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:43.158467 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:43.158473 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:43.158484 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:43.158684 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:43.158684 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:43.158685 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:42:43.158729 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:43.158741 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":96,"threads_started":1}
I20260812 14:42:43.159132 25414 tablet_service.cc:1467] Processing CreateTablet for tablet 758ad15e7e1347efabae42b6b20a1b44 (DEFAULT_TABLE table=test-table [id=d6c852360f20421f9710b9b68a1e365d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:43.159443 25414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758ad15e7e1347efabae42b6b20a1b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:43.161866 26071 tablet_bootstrap.cc:492] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e: Bootstrap starting.
I20260812 14:42:43.162197 25917 tablet_service.cc:1467] Processing CreateTablet for tablet 758ad15e7e1347efabae42b6b20a1b44 (DEFAULT_TABLE table=test-table [id=d6c852360f20421f9710b9b68a1e365d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:43.162462 25917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758ad15e7e1347efabae42b6b20a1b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:43.162659 26071 tablet_bootstrap.cc:654] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:43.162922 26071 log.cc:824] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:43.163586 26071 tablet_bootstrap.cc:492] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e: No bootstrap required, opened a new log
I20260812 14:42:43.163641 26071 ts_tablet_manager.cc:1397] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:43.164842 26079 tablet_bootstrap.cc:492] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Bootstrap starting.
I20260812 14:42:43.165045 26069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.156919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.15.227.194:39201)
0812 14:42:43.157066 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.157071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.157650 (+ 579us) server_negotiation.cc:407] Connection header received
0812 14:42:43.157681 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.157683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.157695 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.157784 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.158564 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.158570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.158571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.158574 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.158604 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.158626 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.158676 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.158733 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.164959 (+ 6226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260812 14:42:43.165575 26079 tablet_bootstrap.cc:654] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:43.165539 25661 tablet_service.cc:1467] Processing CreateTablet for tablet 758ad15e7e1347efabae42b6b20a1b44 (DEFAULT_TABLE table=test-table [id=d6c852360f20421f9710b9b68a1e365d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:43.165800 25661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 758ad15e7e1347efabae42b6b20a1b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:43.165694 26071 raft_consensus.cc:348] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.165845 26071 raft_consensus.cc:374] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:43.165889 26079 log.cc:824] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:43.165913 26071 raft_consensus.cc:729] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e5384fa956f49e99cb8d823d712698e, State: Initialized, Role: FOLLOWER
I20260812 14:42:43.166051 26071 consensus_queue.cc:260] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.166333 26071 ts_tablet_manager.cc:1428] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:42:43.166458 25542 heartbeater.cc:503] Master 127.15.227.254:36899 was elected leader, sending a full tablet report...
I20260812 14:42:43.166608 26079 tablet_bootstrap.cc:492] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: No bootstrap required, opened a new log
I20260812 14:42:43.166682 26079 ts_tablet_manager.cc:1397] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:42:43.168668 26086 tablet_bootstrap.cc:492] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Bootstrap starting.
I20260812 14:42:43.168830 26079 raft_consensus.cc:348] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.168987 26079 raft_consensus.cc:374] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:43.169026 26079 raft_consensus.cc:729] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfaeacf6e49d4bdd837c666a49adfc8f, State: Initialized, Role: FOLLOWER
I20260812 14:42:43.169131 26079 consensus_queue.cc:260] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.169309 26086 tablet_bootstrap.cc:654] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:43.169373 26079 ts_tablet_manager.cc:1428] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:42:43.169387 26049 heartbeater.cc:503] Master 127.15.227.254:36899 was elected leader, sending a full tablet report...
I20260812 14:42:43.169584 26086 log.cc:824] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:43.170315 26086 tablet_bootstrap.cc:492] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: No bootstrap required, opened a new log
I20260812 14:42:43.170378 26086 ts_tablet_manager.cc:1397] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:43.172037 26086 raft_consensus.cc:348] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.172151 26086 raft_consensus.cc:374] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:43.172183 26086 raft_consensus.cc:729] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa42ca15edbe48ac9d425d523d9382ce, State: Initialized, Role: FOLLOWER
I20260812 14:42:43.172292 26086 consensus_queue.cc:260] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.172530 26086 ts_tablet_manager.cc:1428] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:42:43.172606 25775 heartbeater.cc:503] Master 127.15.227.254:36899 was elected leader, sending a full tablet report...
W20260812 14:42:43.280012 25776 tablet.cc:2369] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:43.373808 26085 raft_consensus.cc:482] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:43.373943 26085 raft_consensus.cc:504] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.374321 26085 leader_election.cc:288] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfaeacf6e49d4bdd837c666a49adfc8f (127.15.227.195:38219), fa42ca15edbe48ac9d425d523d9382ce (127.15.227.194:39201)
I20260812 14:42:43.375455 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.374498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:38219 (local address 127.15.227.193:40235)
0812 14:42:43.374563 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.374576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.374617 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.374732 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.374733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.374735 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:43.375135 (+ 400us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:43.375136 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:43.375143 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:43.375154 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:43.375334 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:43.375335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:43.375336 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:42:43.375378 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:42:43.375390 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 14:42:43.375473 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.374588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:40235 (local address 127.15.227.195:38219)
0812 14:42:43.374629 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.374631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.374639 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:43.374666 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.374667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.374673 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.374722 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.375243 (+ 521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.375246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.375246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.375251 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.375271 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.375275 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.375331 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.375385 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.375398 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:43.375540 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.374506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:39201 (local address 127.15.227.193:35891)
0812 14:42:43.374711 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.374720 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.374773 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.374919 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.374920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.374922 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:43.375257 (+ 335us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:42:43.375258 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:43.375263 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:42:43.375272 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:43.375424 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:43.375425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:43.375425 (+ 0us) client_negotiation.cc:769] Sending connection context
0812 14:42:43.375468 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:42:43.375479 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":21,"thread_start_us":120,"threads_started":1}
I20260812 14:42:43.375540 26069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.374588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:35891 (local address 127.15.227.194:39201)
0812 14:42:43.374649 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.374651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.374781 (+ 130us) server_negotiation.cc:407] Connection header received
0812 14:42:43.374842 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.374844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.374849 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.374900 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.375341 (+ 441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.375342 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.375342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.375347 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.375364 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.375369 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.375422 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.375467 (+ 45us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.375479 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:42:43.375757 25953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758ad15e7e1347efabae42b6b20a1b44" candidate_uuid: "1e5384fa956f49e99cb8d823d712698e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" is_pre_election: true
I20260812 14:42:43.375902 25953 raft_consensus.cc:2454] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e5384fa956f49e99cb8d823d712698e in term 0.
I20260812 14:42:43.375887 25691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758ad15e7e1347efabae42b6b20a1b44" candidate_uuid: "1e5384fa956f49e99cb8d823d712698e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa42ca15edbe48ac9d425d523d9382ce" is_pre_election: true
I20260812 14:42:43.376022 25691 raft_consensus.cc:2454] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e5384fa956f49e99cb8d823d712698e in term 0.
I20260812 14:42:43.376127 25340 leader_election.cc:302] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e5384fa956f49e99cb8d823d712698e, cfaeacf6e49d4bdd837c666a49adfc8f; no voters:
I20260812 14:42:43.376247 26085 raft_consensus.cc:2781] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:43.376314 26085 raft_consensus.cc:482] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:43.376330 26085 raft_consensus.cc:3037] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:43.377044 26085 raft_consensus.cc:504] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.377169 26085 leader_election.cc:288] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [CANDIDATE]: Term 1 election: Requested vote from peers cfaeacf6e49d4bdd837c666a49adfc8f (127.15.227.195:38219), fa42ca15edbe48ac9d425d523d9382ce (127.15.227.194:39201)
I20260812 14:42:43.377295 25953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758ad15e7e1347efabae42b6b20a1b44" candidate_uuid: "1e5384fa956f49e99cb8d823d712698e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f"
I20260812 14:42:43.377306 25691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "758ad15e7e1347efabae42b6b20a1b44" candidate_uuid: "1e5384fa956f49e99cb8d823d712698e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa42ca15edbe48ac9d425d523d9382ce"
I20260812 14:42:43.377372 25691 raft_consensus.cc:3037] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:43.377372 25953 raft_consensus.cc:3037] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:43.378281 25691 raft_consensus.cc:2454] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e5384fa956f49e99cb8d823d712698e in term 1.
I20260812 14:42:43.378281 25953 raft_consensus.cc:2454] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e5384fa956f49e99cb8d823d712698e in term 1.
I20260812 14:42:43.378430 25340 leader_election.cc:302] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1e5384fa956f49e99cb8d823d712698e, cfaeacf6e49d4bdd837c666a49adfc8f, fa42ca15edbe48ac9d425d523d9382ce; no voters:
I20260812 14:42:43.378559 26085 raft_consensus.cc:2781] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:43.378706 26085 raft_consensus.cc:686] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 1 LEADER]: Becoming Leader. State: Replica: 1e5384fa956f49e99cb8d823d712698e, State: Running, Role: LEADER
I20260812 14:42:43.378803 26085 consensus_queue.cc:237] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } }
I20260812 14:42:43.379596 25202 catalog_manager.cc:5495] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e5384fa956f49e99cb8d823d712698e (127.15.227.193). New cstate: current_term: 1 leader_uuid: "1e5384fa956f49e99cb8d823d712698e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e5384fa956f49e99cb8d823d712698e" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 43423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:43.382768 26067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.381920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58628 (local address 127.15.227.193:43423)
0812 14:42:43.381959 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.381961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.381968 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:43.382040 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.382041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.382052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.382100 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.382575 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.382576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.382577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.382581 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.382595 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.382599 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.382642 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.382698 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.382710 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:43.384224 25953 raft_consensus.cc:1261] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 1 FOLLOWER]: Refusing update from remote peer 1e5384fa956f49e99cb8d823d712698e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:43.384224 25691 raft_consensus.cc:1261] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 1 FOLLOWER]: Refusing update from remote peer 1e5384fa956f49e99cb8d823d712698e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:43.384541 26095 consensus_queue.cc:1035] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa42ca15edbe48ac9d425d523d9382ce" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:43.384653 26085 consensus_queue.cc:1035] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfaeacf6e49d4bdd837c666a49adfc8f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 38219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:43.386915 26102 mvcc.cc:205] Tried to move back new op lower bound from 7317691446819598336 to 7317691446800138240. Current Snapshot: MvccSnapshot[applied={T|T < 7317691446819598336}]
I20260812 14:42:43.388968 25202 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:43.389041 25202 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54046:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:43.389649 25202 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4936ae361570460ab9894fad3abe34ff: Sending DeleteTablet for 3 replicas of tablet 758ad15e7e1347efabae42b6b20a1b44
I20260812 14:42:43.389925 25414 tablet_service.cc:1514] Processing DeleteTablet for tablet 758ad15e7e1347efabae42b6b20a1b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:43 UTC) from {username='slave'} at 127.0.0.1:58620
I20260812 14:42:43.389951 25917 tablet_service.cc:1514] Processing DeleteTablet for tablet 758ad15e7e1347efabae42b6b20a1b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:43 UTC) from {username='slave'} at 127.0.0.1:51912
I20260812 14:42:43.390174 25661 tablet_service.cc:1514] Processing DeleteTablet for tablet 758ad15e7e1347efabae42b6b20a1b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:43 UTC) from {username='slave'} at 127.0.0.1:60554
I20260812 14:42:43.390264 26116 tablet_replica.cc:330] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e: stopping tablet replica
I20260812 14:42:43.390350 26116 raft_consensus.cc:2229] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:43.390440 26117 tablet_replica.cc:330] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: stopping tablet replica
I20260812 14:42:43.390468 26116 raft_consensus.cc:2258] T 758ad15e7e1347efabae42b6b20a1b44 P 1e5384fa956f49e99cb8d823d712698e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:43.390516 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25314
I20260812 14:42:43.391140 26117 raft_consensus.cc:2229] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:43.391211 26118 tablet_replica.cc:330] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: stopping tablet replica
I20260812 14:42:43.391237 26117 raft_consensus.cc:2258] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:43.391320 26118 raft_consensus.cc:2229] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:43.391433 26118 raft_consensus.cc:2258] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:43.399734 26117 ts_tablet_manager.cc:1905] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:43.399760 26118 ts_tablet_manager.cc:1905] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:43.401187 26117 ts_tablet_manager.cc:1918] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:43.401314 26117 log.cc:1196] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/758ad15e7e1347efabae42b6b20a1b44
I20260812 14:42:43.401486 26118 ts_tablet_manager.cc:1918] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:43.401574 26118 log.cc:1196] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/758ad15e7e1347efabae42b6b20a1b44
I20260812 14:42:43.401664 26117 ts_tablet_manager.cc:1939] T 758ad15e7e1347efabae42b6b20a1b44 P cfaeacf6e49d4bdd837c666a49adfc8f: Deleting consensus metadata
I20260812 14:42:43.401787 26118 ts_tablet_manager.cc:1939] T 758ad15e7e1347efabae42b6b20a1b44 P fa42ca15edbe48ac9d425d523d9382ce: Deleting consensus metadata
I20260812 14:42:43.402251 25178 catalog_manager.cc:4841] TS cfaeacf6e49d4bdd837c666a49adfc8f (127.15.227.195:38219): tablet 758ad15e7e1347efabae42b6b20a1b44 (table test-table [id=d6c852360f20421f9710b9b68a1e365d]) successfully deleted
I20260812 14:42:43.408826 25175 catalog_manager.cc:4841] TS fa42ca15edbe48ac9d425d523d9382ce (127.15.227.194:39201): tablet 758ad15e7e1347efabae42b6b20a1b44 (table test-table [id=d6c852360f20421f9710b9b68a1e365d]) successfully deleted
W20260812 14:42:43.412824 25176 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:43423 (error 108)
I20260812 14:42:43.413286 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.413165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:43423 (local address 127.0.0.1:58636)
0812 14:42:43.413217 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.413259 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:43423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:42:43.413384 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25546
W20260812 14:42:43.413404 25176 catalog_manager.cc:4584] TS 1e5384fa956f49e99cb8d823d712698e (127.15.227.193:43423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 758ad15e7e1347efabae42b6b20a1b44: Network error: Client connection negotiation failed: client connection to 127.15.227.193:43423: connect: Connection refused (error 111)
I20260812 14:42:43.427592 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25780
I20260812 14:42:43.435272 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25147
2026-08-12T14:42:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (821 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 14:42:43.461213 16271 mini_oidc.cc:152] Starting JWKS server
I20260812 14:42:43.462455 16271 webserver.cc:460] Webserver started at https://127.0.0.1:41119/ using document root <none> and password file <none>
I20260812 14:42:43.462551 16271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:42:43.462711 16271 webserver.cc:460] Webserver started at http://127.0.0.1:35387/ using document root <none> and password file <none>
2026-08-12T14:42:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:43Z Disabled control of system clock
I20260812 14:42:43.470558 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34773
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:34773
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41119/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/testchainca.cert with env {}
W20260812 14:42:43.548884 26129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:43.549067 26129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:43.549088 26129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:43.550180 26129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:43.550215 26129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:43.550225 26129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:43.550236 26129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:43.550262 26129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:42:43.550278 26129 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:42:43.550293 26129 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41119/jwks.json
I20260812 14:42:43.551831 26129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:34773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41119/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:43.552021 26129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:43.552178 26129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:43.556576 26141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:43.556579 26143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:43.556588 26139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:43.556736 26129 server_base.cc:1034] running on GCE node
I20260812 14:42:43.557061 26129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:43.557320 26129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:43.558452 26129 hybrid_clock.cc:648] HybridClock initialized: now 1786545763558440 us; error 28 us; skew 500 ppm
I20260812 14:42:43.559528 26129 webserver.cc:460] Webserver started at http://127.15.227.254:34845/ using document root <none> and password file <none>
I20260812 14:42:43.559756 26129 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:43.559811 26129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:43.559959 26129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:43.560878 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "3d971ed6834a4ee5ab9a3c739fdaa128"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "e53f1348f5c6b616732ac21bf29887ed"
server_key_iv: "24c7f48dbb33968f4d7af9940395f180"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.561183 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "3d971ed6834a4ee5ab9a3c739fdaa128"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "e53f1348f5c6b616732ac21bf29887ed"
server_key_iv: "24c7f48dbb33968f4d7af9940395f180"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.562376 26129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:43.563001 26159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.563143 26129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:43.563221 26129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "3d971ed6834a4ee5ab9a3c739fdaa128"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "e53f1348f5c6b616732ac21bf29887ed"
server_key_iv: "24c7f48dbb33968f4d7af9940395f180"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.563287 26129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:43.586864 26129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:43.587129 26129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:43.587268 26129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:43.591040 26129 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:34773
I20260812 14:42:43.591073 26251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:34773 every 8 connection(s)
I20260812 14:42:43.591374 26129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:43.592315 26253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:43.594509 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26129
I20260812 14:42:43.594599 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:43.594764 16271 external_mini_cluster.cc:1428] Setting key cf153962dfec9c3c5900e831d8b2adc7
I20260812 14:42:43.594906 26253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128: Bootstrap starting.
I20260812 14:42:43.595649 26253 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:43.595979 26253 log.cc:824] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:43.596637 26253 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128: No bootstrap required, opened a new log
I20260812 14:42:43.598384 26253 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34773 } }
I20260812 14:42:43.598515 26253 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:43.598546 26253 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d971ed6834a4ee5ab9a3c739fdaa128, State: Initialized, Role: FOLLOWER
I20260812 14:42:43.598671 26253 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34773 } }
I20260812 14:42:43.598743 26253 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:43.598774 26253 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:43.598837 26253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:43.599417 26253 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34773 } }
I20260812 14:42:43.599516 26253 leader_election.cc:302] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d971ed6834a4ee5ab9a3c739fdaa128; no voters:
I20260812 14:42:43.599670 26253 leader_election.cc:288] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:43.599713 26264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:43.599871 26264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [term 1 LEADER]: Becoming Leader. State: Replica: 3d971ed6834a4ee5ab9a3c739fdaa128, State: Running, Role: LEADER
I20260812 14:42:43.600023 26253 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:43.599997 26264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34773 } }
I20260812 14:42:43.600183 26258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.595229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.15.227.254:34773)
0812 14:42:43.595468 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.595476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.595508 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:42:43.596069 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.596077 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.596161 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.596301 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.596499 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.597375 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.598167 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.598569 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.598736 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:43.599345 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.599348 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.599349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.599506 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.599547 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.599562 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.599666 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.599731 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.599899 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260812 14:42:43.600452 26266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d971ed6834a4ee5ab9a3c739fdaa128. Latest consensus state: current_term: 1 leader_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34773 } } }
I20260812 14:42:43.600600 26266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:43.600955 26265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d971ed6834a4ee5ab9a3c739fdaa128" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34773 } } }
I20260812 14:42:43.600979 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:42:43.601028 26265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128 [sys.catalog]: This master's current role is: LEADER
W20260812 14:42:43.602097 26294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:42:43.602154 26294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:42:43.602207 26272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:43.602351 26272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:43.603677 26272 catalog_manager.cc:1347] Generated new cluster ID: dc759d82917642baae8d2904cbc1f77e
I20260812 14:42:43.603727 26272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:43.625907 26272 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:43.626423 26272 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:43.637995 26272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d971ed6834a4ee5ab9a3c739fdaa128: Generated new TSK 0
I20260812 14:42:43.638190 26272 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 14:42:43.663297 26307 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 14:42:43.663446 26307 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 14:42:43.663578 26258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.648739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.15.227.254:34773)
0812 14:42:43.648795 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.648797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.648929 (+ 132us) server_negotiation.cc:407] Connection header received
0812 14:42:43.649030 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.649031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.649040 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.649092 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:42:43.649248 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.649728 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.650861 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.651053 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.651104 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:43.651294 (+ 190us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:42:43.663369 (+ 12075us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 14:42:43.663518 (+ 149us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":32}
W20260812 14:42:43.663705 26258 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41982: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 14:42:43.664060 26306 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:43.648684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:34773 (local address 127.0.0.1:41982)
0812 14:42:43.648863 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.648874 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.648945 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.649110 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.649112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.649114 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:42:43.649187 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:43.649189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.649804 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.649805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:43.650778 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:43.650782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.651122 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.651123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:43.651182 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:43.651205 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:42:43.663529 (+ 12324us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 14:42:43.663641 (+ 112us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.227.254:34773: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.227.254:34773: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41119/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 14:42:43.664459 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26129
2026-08-12T14:42:43Z chronyd exiting
I20260812 14:42:43.707185 16271 test_util.cc:175] -----------------------------------------------
I20260812 14:42:43.707249 16271 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786545740023890-16271-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 14:42:43.707980 16271 mini_oidc.cc:152] Starting JWKS server
I20260812 14:42:43.708617 16271 webserver.cc:460] Webserver started at https://127.0.0.1:40363/ using document root <none> and password file <none>
I20260812 14:42:43.708657 16271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:42:43.708803 16271 webserver.cc:460] Webserver started at http://127.0.0.1:45361/ using document root <none> and password file <none>
2026-08-12T14:42:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:43Z Disabled control of system clock
I20260812 14:42:43.716463 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:32839
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:41701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:32839
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40363/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/testchainca.cert with env {}
W20260812 14:42:43.795215 26318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:43.795401 26318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:43.795431 26318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:43.796533 26318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:43.796572 26318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:43.796588 26318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:43.796603 26318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:43.796618 26318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:42:43.796643 26318 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:42:43.796664 26318 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40363/jwks.json
I20260812 14:42:43.798172 26318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:41701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:32839
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:32839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40363/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:43.798405 26318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:43.798593 26318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:43.803121 26332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:43.803278 26318 server_base.cc:1034] running on GCE node
W20260812 14:42:43.803138 26331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:43.803145 26334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:43.803606 26318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:43.803879 26318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:43.805020 26318 hybrid_clock.cc:648] HybridClock initialized: now 1786545763804996 us; error 35 us; skew 500 ppm
I20260812 14:42:43.806185 26318 webserver.cc:460] Webserver started at http://127.15.227.254:45575/ using document root <none> and password file <none>
I20260812 14:42:43.806393 26318 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:43.806439 26318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:43.806645 26318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:43.807621 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "3e5383818ca44675a6134d9dcaba047a"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "a53851669b837aa48368b18358158271"
server_key_iv: "639680202aa2e3b35413397a2a6124e9"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.807956 26318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "3e5383818ca44675a6134d9dcaba047a"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "a53851669b837aa48368b18358158271"
server_key_iv: "639680202aa2e3b35413397a2a6124e9"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.809172 26318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:43.809826 26342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:43.809988 26318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:43.810093 26318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "3e5383818ca44675a6134d9dcaba047a"
format_stamp: "Formatted at 2026-08-12 14:42:43 on dist-test-slave-q9h9"
server_key: "a53851669b837aa48368b18358158271"
server_key_iv: "639680202aa2e3b35413397a2a6124e9"
server_key_version: "encryptionkey@0"
I20260812 14:42:43.810163 26318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:43.824661 26318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:43.824910 26318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:43.825027 26318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:43.828887 26318 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:32839
I20260812 14:42:43.828915 26448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:32839 every 8 connection(s)
I20260812 14:42:43.829214 26318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:43.829819 26450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:43.830295 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26318
I20260812 14:42:43.830386 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:43.830557 16271 external_mini_cluster.cc:1428] Setting key 8f127b4cb1a9508ea9429ba9723fa85b
I20260812 14:42:43.833514 26450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a: Bootstrap starting.
I20260812 14:42:43.834164 26450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:43.834430 26450 log.cc:824] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:43.835105 26450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a: No bootstrap required, opened a new log
I20260812 14:42:43.836123 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.831055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.15.227.254:32839)
0812 14:42:43.831285 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.831293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.831323 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:42:43.831765 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.831775 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.831854 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.831997 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:43.832142 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.833454 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.834292 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.834712 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.834829 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:43.835466 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:43.835470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:43.835471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:43.835660 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:43.835699 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:43.835714 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:43.835784 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:43.835836 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:42:43.835912 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260812 14:42:43.837060 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:42:43.837184 26450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5383818ca44675a6134d9dcaba047a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32839 } }
I20260812 14:42:43.837325 26450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:43.837359 26450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e5383818ca44675a6134d9dcaba047a, State: Initialized, Role: FOLLOWER
I20260812 14:42:43.837472 26450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5383818ca44675a6134d9dcaba047a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32839 } }
I20260812 14:42:43.837543 26450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:43.837565 26450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:43.837610 26450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:43.838186 26450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5383818ca44675a6134d9dcaba047a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32839 } }
I20260812 14:42:43.838284 26450 leader_election.cc:302] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e5383818ca44675a6134d9dcaba047a; no voters:
I20260812 14:42:43.838426 26450 leader_election.cc:288] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:43.838479 26459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:43.838609 26459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [term 1 LEADER]: Becoming Leader. State: Replica: 3e5383818ca44675a6134d9dcaba047a, State: Running, Role: LEADER
I20260812 14:42:43.838642 26450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:43.838711 26459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5383818ca44675a6134d9dcaba047a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32839 } }
I20260812 14:42:43.839042 26461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e5383818ca44675a6134d9dcaba047a. Latest consensus state: current_term: 1 leader_uuid: "3e5383818ca44675a6134d9dcaba047a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5383818ca44675a6134d9dcaba047a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32839 } } }
I20260812 14:42:43.839174 26461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:43.839032 26460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e5383818ca44675a6134d9dcaba047a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e5383818ca44675a6134d9dcaba047a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 32839 } } }
I20260812 14:42:43.839371 26460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:43.839506 26468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:43.840116 26468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:43.841507 26468 catalog_manager.cc:1347] Generated new cluster ID: 8755ec53cf7848c2b0944e391708b11b
I20260812 14:42:43.841562 26468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:43.847298 26468 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:43.847919 26468 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:43.852497 26468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e5383818ca44675a6134d9dcaba047a: Generated new TSK 0
I20260812 14:42:43.852645 26468 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 14:42:43.873389 26500 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 14:42:43.873461 26500 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 14:42:43.873718 26454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:43.859405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39328 (local address 127.15.227.254:32839)
0812 14:42:43.859445 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:43.859448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:43.859536 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:42:43.859601 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:43.859602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:43.859611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:43.859663 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:42:43.859769 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.860216 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.861283 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.861477 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.861515 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:43.861680 (+ 165us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:42:43.873554 (+ 11874us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40363/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 14:42:43.873671 (+ 117us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39328: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40363/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:42:43.873764 26454 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39328: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40363/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260812 14:42:43.873873 26499 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:43.859339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:32839 (local address 127.0.0.1:39328)
0812 14:42:43.859470 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:43.859480 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:43.859544 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:43.859664 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:43.859665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:43.859666 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:42:43.859719 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:43.859721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.860287 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.860288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:43.861215 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:43.861218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:43.861535 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:43.861536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:43.861601 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:43.861606 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:42:43.873656 (+ 12050us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40363/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 14:42:43.873776 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.227.254:32839: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40363/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260812 14:42:43.874138 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26318
2026-08-12T14:42:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (212 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 14:42:43.920120 16271 mini_oidc.cc:152] Starting JWKS server
I20260812 14:42:43.920778 16271 webserver.cc:460] Webserver started at https://127.0.0.1:43201/ using document root <none> and password file <none>
I20260812 14:42:43.920828 16271 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 14:42:43.920969 16271 webserver.cc:460] Webserver started at http://127.0.0.1:33779/ using document root <none> and password file <none>
2026-08-12T14:42:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:43Z Disabled control of system clock
I20260812 14:42:43.928717 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43087
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:43087
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43201/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 14:42:44.006942 26514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:44.007138 26514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:44.007166 26514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:44.008285 26514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:44.008323 26514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:44.008340 26514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:44.008356 26514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:44.008371 26514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:42:44.008387 26514 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:42:44.008406 26514 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43201/jwks.json
I20260812 14:42:44.009743 26514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:43087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43201/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:44.009974 26514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:44.010248 26514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:44.014874 26528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:44.014940 26532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:44.014940 26537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:44.015147 26514 server_base.cc:1034] running on GCE node
I20260812 14:42:44.015425 26514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:44.015745 26514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:44.016929 26514 hybrid_clock.cc:648] HybridClock initialized: now 1786545764016921 us; error 36 us; skew 500 ppm
I20260812 14:42:44.018344 26514 webserver.cc:460] Webserver started at http://127.15.227.254:34987/ using document root <none> and password file <none>
I20260812 14:42:44.018573 26514 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:44.018621 26514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:44.018744 26514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:44.019855 26514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "4c20fa338e764f8da984778d3a28cbc1"
format_stamp: "Formatted at 2026-08-12 14:42:44 on dist-test-slave-q9h9"
server_key: "437db0b93fa092a2286502883ef76e15"
server_key_iv: "bb80e483dfddaa158850eacf7d84cb23"
server_key_version: "encryptionkey@0"
I20260812 14:42:44.020196 26514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "4c20fa338e764f8da984778d3a28cbc1"
format_stamp: "Formatted at 2026-08-12 14:42:44 on dist-test-slave-q9h9"
server_key: "437db0b93fa092a2286502883ef76e15"
server_key_iv: "bb80e483dfddaa158850eacf7d84cb23"
server_key_version: "encryptionkey@0"
I20260812 14:42:44.021570 26514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:44.022439 26543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:44.022645 26514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:44.022749 26514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "4c20fa338e764f8da984778d3a28cbc1"
format_stamp: "Formatted at 2026-08-12 14:42:44 on dist-test-slave-q9h9"
server_key: "437db0b93fa092a2286502883ef76e15"
server_key_iv: "bb80e483dfddaa158850eacf7d84cb23"
server_key_version: "encryptionkey@0"
I20260812 14:42:44.022822 26514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:44.039930 26514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:44.040266 26514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:44.040411 26514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:44.044719 26514 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:43087
I20260812 14:42:44.044759 26629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:43087 every 8 connection(s)
I20260812 14:42:44.045109 26514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:44.045770 26636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:44.048516 26636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1: Bootstrap starting.
I20260812 14:42:44.049052 26636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:44.049293 26636 log.cc:824] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:44.049893 26636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1: No bootstrap required, opened a new log
I20260812 14:42:44.051434 26636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c20fa338e764f8da984778d3a28cbc1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43087 } }
I20260812 14:42:44.051565 26636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:44.051587 26636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c20fa338e764f8da984778d3a28cbc1, State: Initialized, Role: FOLLOWER
I20260812 14:42:44.051690 26636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c20fa338e764f8da984778d3a28cbc1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43087 } }
I20260812 14:42:44.051738 26636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:44.051764 26636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:44.051793 26636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:44.052274 26636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c20fa338e764f8da984778d3a28cbc1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43087 } }
I20260812 14:42:44.052353 26636 leader_election.cc:302] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c20fa338e764f8da984778d3a28cbc1; no voters:
I20260812 14:42:44.052466 26636 leader_election.cc:288] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:44.052517 26641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:44.052698 26636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:44.052701 26641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [term 1 LEADER]: Becoming Leader. State: Replica: 4c20fa338e764f8da984778d3a28cbc1, State: Running, Role: LEADER
I20260812 14:42:44.052750 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26514
I20260812 14:42:44.052851 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:44.052884 26641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c20fa338e764f8da984778d3a28cbc1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43087 } }
I20260812 14:42:44.053069 16271 external_mini_cluster.cc:1428] Setting key 69579a93158ab888024f28a214dd443f
I20260812 14:42:44.053221 26645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c20fa338e764f8da984778d3a28cbc1. Latest consensus state: current_term: 1 leader_uuid: "4c20fa338e764f8da984778d3a28cbc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c20fa338e764f8da984778d3a28cbc1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43087 } } }
I20260812 14:42:44.053367 26645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:44.053639 26664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:44.054096 26642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c20fa338e764f8da984778d3a28cbc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c20fa338e764f8da984778d3a28cbc1" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43087 } } }
I20260812 14:42:44.054240 26642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:44.054638 26664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:44.056025 26664 catalog_manager.cc:1347] Generated new cluster ID: 5318cc740f894e599c89144c29d9a2b7
I20260812 14:42:44.056073 26664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:44.059772 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:44.053699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.15.227.254:43087)
0812 14:42:44.054349 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:44.054359 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:44.054396 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:42:44.054509 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:44.054516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:44.054574 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:44.054692 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:44.054969 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.056331 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.057387 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.057999 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.058151 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:44.059002 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:44.059006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:44.059006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:44.059201 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:44.059240 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:44.059254 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:44.059316 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:44.059387 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:42:44.059540 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":84,"threads_started":1}
I20260812 14:42:44.060585 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:42:44.065349 26664 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:44.065807 26664 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:44.069351 26664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c20fa338e764f8da984778d3a28cbc1: Generated new TSK 0
I20260812 14:42:44.069487 26664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:42:44.081151 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:44.078047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41674 (local address 127.15.227.254:43087)
0812 14:42:44.078080 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:44.078082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:44.078089 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:44.078120 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:44.078121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:44.078132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:44.078185 (+ 53us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:42:44.078310 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.078782 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.079869 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.080057 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.080097 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:44.080261 (+ 164us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:42:44.080974 (+ 713us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 14:42:44.081091 (+ 117us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 14:42:44.081202 26665 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 14:42:44.081270 26693 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:44.077820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43087 (local address 127.0.0.1:41674)
0812 14:42:44.077978 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:44.077989 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:44.078067 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:44.078200 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:44.078201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:44.078203 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:42:44.078256 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:44.078258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.078862 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.078863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:44.079797 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:44.079799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.080112 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.080113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:44.080178 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:44.080183 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:42:44.081069 (+ 886us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 14:42:44.081156 (+ 87us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.227.254:43087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43201/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260812 14:42:44.081413 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26514
E20260812 14:42:44.102480 26694 webserver.cc:533] Webserver: sslize failed:
E20260812 14:42:44.102559 26694 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-12T14:42:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T14:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:44Z Disabled control of system clock
I20260812 14:42:44.141064 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36525
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:44113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:36525
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/testchainca.cert with env {}
W20260812 14:42:44.220109 26699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:44.220307 26699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:44.220341 26699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:44.221454 26699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:44.221493 26699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:44.221510 26699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:44.221526 26699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:44.221544 26699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 14:42:44.221563 26699 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 14:42:44.221586 26699 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 14:42:44.223057 26699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:36525
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:44.223299 26699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:44.223491 26699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:44.228188 26708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:44.228236 26716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:44.228273 26711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:44.228425 26699 server_base.cc:1034] running on GCE node
I20260812 14:42:44.228652 26699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:44.228894 26699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:44.230049 26699 hybrid_clock.cc:648] HybridClock initialized: now 1786545764230029 us; error 34 us; skew 500 ppm
I20260812 14:42:44.231232 26699 webserver.cc:460] Webserver started at http://127.15.227.254:46479/ using document root <none> and password file <none>
I20260812 14:42:44.231462 26699 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:44.231504 26699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:44.231639 26699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:44.232631 26699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "76e1e1de255545438b9fffd03ff53fae"
format_stamp: "Formatted at 2026-08-12 14:42:44 on dist-test-slave-q9h9"
server_key: "8d9c68c7d6e331beb43ed22d42c7584a"
server_key_iv: "7a2ae4b7504e5d61acb1bf6d47384e2d"
server_key_version: "encryptionkey@0"
I20260812 14:42:44.232947 26699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "76e1e1de255545438b9fffd03ff53fae"
format_stamp: "Formatted at 2026-08-12 14:42:44 on dist-test-slave-q9h9"
server_key: "8d9c68c7d6e331beb43ed22d42c7584a"
server_key_iv: "7a2ae4b7504e5d61acb1bf6d47384e2d"
server_key_version: "encryptionkey@0"
I20260812 14:42:44.234148 26699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:44.234822 26726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:44.234963 26699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:44.235046 26699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "76e1e1de255545438b9fffd03ff53fae"
format_stamp: "Formatted at 2026-08-12 14:42:44 on dist-test-slave-q9h9"
server_key: "8d9c68c7d6e331beb43ed22d42c7584a"
server_key_iv: "7a2ae4b7504e5d61acb1bf6d47384e2d"
server_key_version: "encryptionkey@0"
I20260812 14:42:44.235119 26699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:44.250880 26699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:44.251168 26699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:44.251294 26699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:44.255478 26699 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:36525
I20260812 14:42:44.255540 26804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:36525 every 8 connection(s)
I20260812 14:42:44.255906 26699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:44.256999 26805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:44.259308 26805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae: Bootstrap starting.
I20260812 14:42:44.259861 26805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:44.260120 26805 log.cc:824] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:44.260766 26805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae: No bootstrap required, opened a new log
I20260812 14:42:44.262357 26805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e1e1de255545438b9fffd03ff53fae" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36525 } }
I20260812 14:42:44.262488 26805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:44.262511 26805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e1e1de255545438b9fffd03ff53fae, State: Initialized, Role: FOLLOWER
I20260812 14:42:44.262629 26805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e1e1de255545438b9fffd03ff53fae" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36525 } }
I20260812 14:42:44.262689 26805 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:44.262708 26805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:44.262737 26805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:44.263221 26805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e1e1de255545438b9fffd03ff53fae" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36525 } }
I20260812 14:42:44.263307 26805 leader_election.cc:302] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76e1e1de255545438b9fffd03ff53fae; no voters:
I20260812 14:42:44.263450 26805 leader_election.cc:288] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:44.263499 26813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:44.263672 26805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:44.263680 26813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [term 1 LEADER]: Becoming Leader. State: Replica: 76e1e1de255545438b9fffd03ff53fae, State: Running, Role: LEADER
I20260812 14:42:44.263886 26813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e1e1de255545438b9fffd03ff53fae" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36525 } }
I20260812 14:42:44.264238 26815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76e1e1de255545438b9fffd03ff53fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e1e1de255545438b9fffd03ff53fae" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36525 } } }
I20260812 14:42:44.264326 26817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76e1e1de255545438b9fffd03ff53fae. Latest consensus state: current_term: 1 leader_uuid: "76e1e1de255545438b9fffd03ff53fae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76e1e1de255545438b9fffd03ff53fae" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36525 } } }
I20260812 14:42:44.264406 26817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:44.264371 26815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:44.264786 26830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:44.265265 26830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:44.265307 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26699
I20260812 14:42:44.265399 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:44.265575 16271 external_mini_cluster.cc:1428] Setting key a7b642edfcc91b949e14f80768ed7260
I20260812 14:42:44.266870 26830 catalog_manager.cc:1347] Generated new cluster ID: e1390576eeba40f4baf796d499a160c2
I20260812 14:42:44.266921 26830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:44.271754 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:44.266199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.15.227.254:36525)
0812 14:42:44.266505 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:44.266514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:44.266545 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:42:44.266653 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:44.266659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:44.266729 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:44.266948 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:44.267174 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.268567 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.269590 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.270232 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.270370 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:44.270968 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:44.270972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:44.270973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:42:44.271172 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:44.271214 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:42:44.271228 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:44.271306 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:44.271358 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:42:44.271435 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260812 14:42:44.272598 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:42:44.289350 26830 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:44.289824 26830 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:44.298151 26830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76e1e1de255545438b9fffd03ff53fae: Generated new TSK 0
I20260812 14:42:44.298321 26830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:42:44.772032 26848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:44.305013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45222 (local address 127.15.227.254:36525)
0812 14:42:44.305064 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:44.305067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:44.305150 (+ 83us) server_negotiation.cc:407] Connection header received
0812 14:42:44.305199 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:44.305200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:44.305224 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:44.305305 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0812 14:42:44.305396 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.306160 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.306923 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.307225 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.307297 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:44.307394 (+ 97us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 14:42:44.771810 (+464416us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 14:42:44.771970 (+ 160us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45222: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":29}
W20260812 14:42:44.772095 26848 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45222: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 14:42:44.772184 26862 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:44.304941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36525 (local address 127.0.0.1:45222)
0812 14:42:44.305093 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:44.305101 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:44.305151 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:44.305302 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:44.305303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:44.305305 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0812 14:42:44.305356 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:44.305358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.306256 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.306257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:44.306856 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:44.306858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:44.307292 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:44.307293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:44.307327 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:44.307331 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 14:42:44.771952 (+464621us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 14:42:44.772078 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.227.254:36525: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260812 14:42:44.772467 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26699
2026-08-12T14:42:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (667 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:45 dist-test-slave-q9h9 krb5kdc[26887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:45 dist-test-slave-q9h9 krb5kdc[26887](info): set up 2 sockets
Aug 12 14:42:45 dist-test-slave-q9h9 krb5kdc[26887](info): commencing operation
krb5kdc: starting...
W20260812 14:42:46.148348 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:46 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:46.254099 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:41201
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:41201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:46.333925 26907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:46.334137 26907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:46.334172 26907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:46.335315 26907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:46.335362 26907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:46.335386 26907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:46.335404 26907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:46.335424 26907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:46.336872 26907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:41201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:41201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:46.337116 26907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:46.337327 26907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:46.341892 26914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:46.341898 26913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:46.341960 26920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:46.341990 26907 server_base.cc:1034] running on GCE node
I20260812 14:42:46.342388 26907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:46.342623 26907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:46.343770 26907 hybrid_clock.cc:648] HybridClock initialized: now 1786545766343759 us; error 29 us; skew 500 ppm
Aug 12 14:42:46 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:46.346153 26907 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:46.346550 26907 webserver.cc:460] Webserver started at http://127.15.227.254:35171/ using document root <none> and password file <none>
I20260812 14:42:46.346778 26907 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:46.346823 26907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:46.346940 26907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:46.347913 26907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "f46b70b9ccac4fd1b61be005f88fe31d"
format_stamp: "Formatted at 2026-08-12 14:42:46 on dist-test-slave-q9h9"
server_key: "5bc11a5a4a164897f010627f376870f0"
server_key_iv: "1984a4dcb5884721388f277c23737d27"
server_key_version: "encryptionkey@0"
I20260812 14:42:46.348220 26907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "f46b70b9ccac4fd1b61be005f88fe31d"
format_stamp: "Formatted at 2026-08-12 14:42:46 on dist-test-slave-q9h9"
server_key: "5bc11a5a4a164897f010627f376870f0"
server_key_iv: "1984a4dcb5884721388f277c23737d27"
server_key_version: "encryptionkey@0"
I20260812 14:42:46.349356 26907 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:46.349967 26929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:46.350114 26907 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:46.350216 26907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "f46b70b9ccac4fd1b61be005f88fe31d"
format_stamp: "Formatted at 2026-08-12 14:42:46 on dist-test-slave-q9h9"
server_key: "5bc11a5a4a164897f010627f376870f0"
server_key_iv: "1984a4dcb5884721388f277c23737d27"
server_key_version: "encryptionkey@0"
I20260812 14:42:46.350274 26907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:46.368438 26907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:46.369918 26907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:46.370066 26907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:46.373564 26907 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:41201
I20260812 14:42:46.373602 27040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:41201 every 8 connection(s)
I20260812 14:42:46.373901 26907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:46.374680 27041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:46.376660 27041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d: Bootstrap starting.
I20260812 14:42:46.377177 27041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:46.377450 27041 log.cc:824] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:46.378031 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26907
I20260812 14:42:46.378151 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:46.378209 27041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d: No bootstrap required, opened a new log
I20260812 14:42:46.378362 16271 external_mini_cluster.cc:1428] Setting key 71eb3070603c62bdda3a48551d425ada
I20260812 14:42:46.380296 27041 raft_consensus.cc:348] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41201 } }
I20260812 14:42:46.380426 27041 raft_consensus.cc:374] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:46.380447 27041 raft_consensus.cc:729] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46b70b9ccac4fd1b61be005f88fe31d, State: Initialized, Role: FOLLOWER
I20260812 14:42:46.380544 27041 consensus_queue.cc:260] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41201 } }
I20260812 14:42:46.380606 27041 raft_consensus.cc:388] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:46.380627 27041 raft_consensus.cc:482] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:46.380671 27041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:46.381150 27041 raft_consensus.cc:504] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41201 } }
I20260812 14:42:46.381228 27041 leader_election.cc:302] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f46b70b9ccac4fd1b61be005f88fe31d; no voters:
I20260812 14:42:46.381393 27041 leader_election.cc:288] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:46.381420 27050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:46.381570 27050 raft_consensus.cc:686] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [term 1 LEADER]: Becoming Leader. State: Replica: f46b70b9ccac4fd1b61be005f88fe31d, State: Running, Role: LEADER
I20260812 14:42:46.381641 27041 sys_catalog.cc:564] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:46.381762 27050 consensus_queue.cc:237] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41201 } }
I20260812 14:42:46.382097 27050 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f46b70b9ccac4fd1b61be005f88fe31d. Latest consensus state: current_term: 1 leader_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41201 } } }
I20260812 14:42:46.382337 27050 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:46.382262 27053 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46b70b9ccac4fd1b61be005f88fe31d" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41201 } } }
I20260812 14:42:46.382553 27053 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:46.382725 27064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 14:42:46 dist-test-slave-q9h9 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:46.383249 27064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:46.388150 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:46.378822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50070 (local address 127.15.227.254:41201)
0812 14:42:46.378998 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:46.379006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:46.379029 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:42:46.379440 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:46.379446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:46.379489 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:46.379592 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:46.380307 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:46.381531 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:46.382504 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:46.383030 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:46.383385 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:46.385230 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:46.385234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:46.385234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:46.385245 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:46.386789 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:46.387093 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:46.387097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:46.387097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:46.387190 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:46.387394 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:46.387395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:46.387396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:46.387599 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:46.387779 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:46.387844 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:42:46.388007 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":82,"threads_started":1}
I20260812 14:42:46.392022 27064 catalog_manager.cc:1347] Generated new cluster ID: 2636124a4cc84bdfa709f2cd1affa190
I20260812 14:42:46.392068 27064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:46.404275 27064 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:46.404760 27064 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:46.410928 27064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f46b70b9ccac4fd1b61be005f88fe31d: Generated new TSK 0
I20260812 14:42:46.411077 27064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
W20260812 14:42:47.177070 16271 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.227.193: real 0.759s user 0.002s sys 0.001s
I20260812 14:42:47.177413 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41201
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:47.262706 27085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:47.262895 27085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:47.262917 27085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:47.264328 27085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:47.264384 27085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:47.264421 27085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:47.266536 27085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:47.266784 27085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:47.267009 27085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:47.271831 27094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:47.271929 27085 server_base.cc:1034] running on GCE node
W20260812 14:42:47.271808 27097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:47.271827 27093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:47.272290 27085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:47.272508 27085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:47.273643 27085 hybrid_clock.cc:648] HybridClock initialized: now 1786545767273628 us; error 28 us; skew 500 ppm
Aug 12 14:42:47 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:47.276089 27085 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:47.276512 27085 webserver.cc:460] Webserver started at http://127.15.227.193:41495/ using document root <none> and password file <none>
I20260812 14:42:47.276728 27085 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:47.276777 27085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:47.276898 27085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:47.277884 27085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "22c3068d744340f8bc9e01e29a9bd2d1"
format_stamp: "Formatted at 2026-08-12 14:42:47 on dist-test-slave-q9h9"
server_key: "fd45c164e1448c2e7dfe0f3065f40bbf"
server_key_iv: "ac54dc7f92273b0c560ce482ff830cfb"
server_key_version: "encryptionkey@0"
I20260812 14:42:47.278226 27085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "22c3068d744340f8bc9e01e29a9bd2d1"
format_stamp: "Formatted at 2026-08-12 14:42:47 on dist-test-slave-q9h9"
server_key: "fd45c164e1448c2e7dfe0f3065f40bbf"
server_key_iv: "ac54dc7f92273b0c560ce482ff830cfb"
server_key_version: "encryptionkey@0"
I20260812 14:42:47.279489 27085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:47.280217 27112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:47.280447 27085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:47.280535 27085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "22c3068d744340f8bc9e01e29a9bd2d1"
format_stamp: "Formatted at 2026-08-12 14:42:47 on dist-test-slave-q9h9"
server_key: "fd45c164e1448c2e7dfe0f3065f40bbf"
server_key_iv: "ac54dc7f92273b0c560ce482ff830cfb"
server_key_version: "encryptionkey@0"
I20260812 14:42:47.280596 27085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:47.290369 27085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:47.292160 27085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:47.292284 27085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:47.292493 27085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:47.292778 27085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:47.292811 27085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:47.292837 27085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:47.292860 27085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:47.298353 27085 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:43649
I20260812 14:42:47.298413 27317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:43649 every 8 connection(s)
I20260812 14:42:47.298679 27085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:47.302706 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 27085
I20260812 14:42:47.302812 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:47.303009 16271 external_mini_cluster.cc:1428] Setting key d76feb4ecb6ea60457d4251a4fde2195
Aug 12 14:42:47 dist-test-slave-q9h9 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:47.308856 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:47.299542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:51929 (local address 127.15.227.254:41201)
0812 14:42:47.299664 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:47.299666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:47.299677 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:47.300553 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:47.300555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:47.300568 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:47.300639 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:47.301264 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:47.301783 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:47.303066 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:47.303333 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:47.303398 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:47.305694 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:47.305696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:47.305696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:47.305705 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:47.307431 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:47.307918 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:47.307919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:47.307919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:47.307974 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:47.308264 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:47.308265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:47.308265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:47.308369 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:47.308416 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:47.308714 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 14:42:47.308784 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260812 14:42:47.309096 27325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:47.299365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41201 (local address 127.15.227.193:51929)
0812 14:42:47.299520 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:47.299559 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:47.300435 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:47.300705 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:47.300709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:47.300944 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:47.301121 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:47.301125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:47.301890 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:47.301891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:47.302975 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:47.302979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:47.303412 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:47.303413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:47.303564 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:47.304073 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:47.304082 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:47.305586 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:47.307522 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:47.307530 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:47.307532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:47.307843 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:47.308027 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:47.308028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:47.308029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:47.308201 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:47.308426 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:47.308432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:47.308590 (+ 158us) client_negotiation.cc:769] Sending connection context
0812 14:42:47.308702 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:42:47.308869 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260812 14:42:47.309569 27318 heartbeater.cc:347] Connected to a master server at 127.15.227.254:41201
I20260812 14:42:47.309670 27318 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:47.309841 27318 heartbeater.cc:511] Master 127.15.227.254:41201 requested a full tablet report, sending...
I20260812 14:42:47.310364 26960 ts_manager.cc:194] Registered new tserver with Master: 22c3068d744340f8bc9e01e29a9bd2d1 (127.15.227.193:43649)
I20260812 14:42:47.310971 26960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:51929
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
W20260812 14:42:48.060606 16271 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.15.227.194: real 0.757s user 0.000s sys 0.004s
I20260812 14:42:48.060979 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41201
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:48.142288 27331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:48.142488 27331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:48.142510 27331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:48.143616 27331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:48.143654 27331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:48.143678 27331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:48.145135 27331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:48.145315 27331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:48.145499 27331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:48.150239 27338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:48.150250 27346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:48.150384 27331 server_base.cc:1034] running on GCE node
W20260812 14:42:48.150260 27342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:48.150667 27331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:48.150884 27331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:48.152046 27331 hybrid_clock.cc:648] HybridClock initialized: now 1786545768152031 us; error 28 us; skew 500 ppm
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.157459 27331 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:48.157848 27331 webserver.cc:460] Webserver started at http://127.15.227.194:40445/ using document root <none> and password file <none>
I20260812 14:42:48.158092 27331 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:48.158143 27331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:48.158257 27331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:48.159220 27331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "cca62fab21c84f6e9e719460d28c8155"
format_stamp: "Formatted at 2026-08-12 14:42:48 on dist-test-slave-q9h9"
server_key: "35f91e1fcb87176a666548f21282b2d6"
server_key_iv: "e93d9f731c9a53330fd5a1d227172a9b"
server_key_version: "encryptionkey@0"
I20260812 14:42:48.159535 27331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "cca62fab21c84f6e9e719460d28c8155"
format_stamp: "Formatted at 2026-08-12 14:42:48 on dist-test-slave-q9h9"
server_key: "35f91e1fcb87176a666548f21282b2d6"
server_key_iv: "e93d9f731c9a53330fd5a1d227172a9b"
server_key_version: "encryptionkey@0"
I20260812 14:42:48.160741 27331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:48.161405 27363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:48.161547 27331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:48.161628 27331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "cca62fab21c84f6e9e719460d28c8155"
format_stamp: "Formatted at 2026-08-12 14:42:48 on dist-test-slave-q9h9"
server_key: "35f91e1fcb87176a666548f21282b2d6"
server_key_iv: "e93d9f731c9a53330fd5a1d227172a9b"
server_key_version: "encryptionkey@0"
I20260812 14:42:48.161692 27331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:48.172912 27331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:48.174495 27331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:48.174616 27331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:48.174821 27331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:48.175091 27331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:48.175123 27331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:48.175150 27331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:48.175168 27331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:48.180428 27331 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:40931
I20260812 14:42:48.180462 27578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:40931 every 8 connection(s)
I20260812 14:42:48.180739 27331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:48.186105 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 27331
I20260812 14:42:48.186219 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:48.186424 16271 external_mini_cluster.cc:1428] Setting key 1fd33435e1ad3d404c4f62d838a898fc
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:48.189886 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:48.181379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:50321 (local address 127.15.227.254:41201)
0812 14:42:48.181504 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.181506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.181624 (+ 118us) server_negotiation.cc:407] Connection header received
0812 14:42:48.182596 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.182597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.182613 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.182676 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.183316 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.183817 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.184825 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.185023 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.185078 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:48.187230 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:48.187231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:48.187232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:48.187255 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:48.188535 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.189003 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.189004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.189005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.189067 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.189307 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.189307 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.189308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.189403 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:48.189448 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:48.189740 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 14:42:48.189811 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260812 14:42:48.190104 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:48.181344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41201 (local address 127.15.227.194:50321)
0812 14:42:48.181510 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:48.181551 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:48.182503 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:48.182755 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:48.182759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:48.183039 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:48.183229 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:48.183234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.183896 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.183897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:48.184737 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:48.184739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.185089 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.185090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:48.185206 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:48.185579 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:48.185587 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:48.187146 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:48.188608 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:48.188615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:48.188617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:48.188898 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:48.189123 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:48.189123 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:48.189124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:48.189250 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:48.189453 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:48.189457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:48.189603 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 14:42:48.189711 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:42:48.189873 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260812 14:42:48.191147 27580 heartbeater.cc:347] Connected to a master server at 127.15.227.254:41201
I20260812 14:42:48.191255 27580 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:48.191437 27580 heartbeater.cc:511] Master 127.15.227.254:41201 requested a full tablet report, sending...
I20260812 14:42:48.191817 26960 ts_manager.cc:194] Registered new tserver with Master: cca62fab21c84f6e9e719460d28c8155 (127.15.227.194:40931)
I20260812 14:42:48.192171 26960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:50321
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:48.254989 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41201
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:42:48.311787 27318 heartbeater.cc:503] Master 127.15.227.254:41201 was elected leader, sending a full tablet report...
W20260812 14:42:48.335937 27587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:48.336123 27587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:48.336145 27587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:48.337257 27587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:48.337296 27587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:48.337325 27587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:48.338862 27587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:41201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:48.339078 27587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:48.339277 27587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:48.343865 27593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:48.343974 27595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:48.343899 27592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:48.344025 27587 server_base.cc:1034] running on GCE node
I20260812 14:42:48.344326 27587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:48.344532 27587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:48.345678 27587 hybrid_clock.cc:648] HybridClock initialized: now 1786545768345662 us; error 28 us; skew 500 ppm
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.348158 27587 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:48.348595 27587 webserver.cc:460] Webserver started at http://127.15.227.195:36837/ using document root <none> and password file <none>
I20260812 14:42:48.348798 27587 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:48.348841 27587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:48.348963 27587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:48.349918 27587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "bcc62e42dc8545a79592c0a09f5a8ebd"
format_stamp: "Formatted at 2026-08-12 14:42:48 on dist-test-slave-q9h9"
server_key: "894e3bed018f8e529bef1a30913ae5d0"
server_key_iv: "5595ba76d3fc5c083915a55ceb680d4c"
server_key_version: "encryptionkey@0"
I20260812 14:42:48.350302 27587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "bcc62e42dc8545a79592c0a09f5a8ebd"
format_stamp: "Formatted at 2026-08-12 14:42:48 on dist-test-slave-q9h9"
server_key: "894e3bed018f8e529bef1a30913ae5d0"
server_key_iv: "5595ba76d3fc5c083915a55ceb680d4c"
server_key_version: "encryptionkey@0"
I20260812 14:42:48.351504 27587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:48.352133 27607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:48.352304 27587 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:48.352406 27587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "bcc62e42dc8545a79592c0a09f5a8ebd"
format_stamp: "Formatted at 2026-08-12 14:42:48 on dist-test-slave-q9h9"
server_key: "894e3bed018f8e529bef1a30913ae5d0"
server_key_iv: "5595ba76d3fc5c083915a55ceb680d4c"
server_key_version: "encryptionkey@0"
I20260812 14:42:48.352484 27587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:48.371217 27587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:48.372874 27587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:48.372979 27587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:48.373155 27587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:48.373404 27587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:48.373431 27587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:48.373451 27587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:48.373464 27587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:48.379411 27587 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:43115
I20260812 14:42:48.379477 27806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:43115 every 8 connection(s)
I20260812 14:42:48.379725 27587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:48.388053 27584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:48.380388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:32875 (local address 127.15.227.254:41201)
0812 14:42:48.380440 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.380442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.380641 (+ 199us) server_negotiation.cc:407] Connection header received
0812 14:42:48.381565 (+ 924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.381567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.381577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.381617 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.382249 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.382791 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.383829 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.384018 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.384066 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:48.385987 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:48.385988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:48.385989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:48.385995 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:48.387068 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.387392 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.387393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.387393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.387436 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.387611 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.387611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.387611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.387702 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:48.387746 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:48.387930 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:42:48.387990 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:42:48.388222 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:48.380346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41201 (local address 127.15.227.195:32875)
0812 14:42:48.380513 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:48.380566 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:48.381466 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:48.381674 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:48.381677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:48.381913 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:48.382139 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:48.382144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.382858 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.382859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:48.383766 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:48.383769 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.384076 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.384076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:48.384166 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:48.384633 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:48.384643 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:48.385908 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:48.387130 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:48.387136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:48.387137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:48.387334 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:48.387476 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:48.387476 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:48.387477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:48.387570 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:48.387741 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:48.387744 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:48.387843 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 14:42:48.387914 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:42:48.388020 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260812 14:42:48.388696 27807 heartbeater.cc:347] Connected to a master server at 127.15.227.254:41201
I20260812 14:42:48.388795 27807 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:48.388957 27807 heartbeater.cc:511] Master 127.15.227.254:41201 requested a full tablet report, sending...
I20260812 14:42:48.389289 26960 ts_manager.cc:194] Registered new tserver with Master: bcc62e42dc8545a79592c0a09f5a8ebd (127.15.227.195:43115)
I20260812 14:42:48.389619 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 27587
I20260812 14:42:48.389632 26960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:32875
I20260812 14:42:48.389725 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:48.389914 16271 external_mini_cluster.cc:1428] Setting key a36411c72ba5a478b1c5301abb10cffa
I20260812 14:42:48.390671 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/test-admin.keytab.
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.406450 16271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 14:42:48.407963 27819 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:48.407284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.15.227.193:43649)
0812 14:42:48.407422 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.407427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.407435 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:48.407458 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.407459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.407472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.407532 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.407887 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36232: BlockingRecv error: recv got EOF from 127.0.0.1:36232 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":57,"threads_started":1}
W20260812 14:42:48.408005 16271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.408885 16271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.409714 16271 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:48.416867 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:48.410299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36236 (local address 127.15.227.193:43649)
0812 14:42:48.410345 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.410347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.410355 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:48.410404 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.410405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.410415 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.410472 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.410840 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.411520 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.412210 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.412521 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.412578 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:48.414548 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:48.414555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:48.414556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:48.414563 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:48.415837 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.416145 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.416149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.416150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.416207 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.416367 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.416368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.416368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.416503 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:48.416620 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:48.416709 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 14:42:48.416781 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260812 14:42:48.418452 27819 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:48.417945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36248 (local address 127.15.227.193:43649)
0812 14:42:48.417980 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.417982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.418006 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:42:48.418096 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.418097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.418107 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.418150 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.418410 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36248: BlockingRecv error: recv got EOF from 127.0.0.1:36248 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:42:48.418489 16271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.419299 16271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.420006 16271 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:48.425669 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:48.420421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36252 (local address 127.15.227.193:43649)
0812 14:42:48.420459 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.420461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.420485 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:42:48.420533 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.420534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.420540 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.420582 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.420864 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.421306 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.421897 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.422127 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.422173 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:48.423880 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:48.423882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:48.423882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:48.423888 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:48.424891 (+ 1003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.425161 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.425161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.425162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.425211 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.425346 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.425346 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.425346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.425419 (+ 73us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:48.425478 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:48.425552 (+ 74us) server_negotiation.cc:299] Negotiation successful
0812 14:42:48.425606 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:42:48.427094 27819 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:48.426607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36266 (local address 127.15.227.193:43649)
0812 14:42:48.426650 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.426651 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.426659 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:48.426705 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.426706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.426712 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.426751 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.427058 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36266: BlockingRecv error: recv got EOF from 127.0.0.1:36266 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:42:48.427179 16271 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786545740023890-16271-0/krb5kdc/krb5cc' of type FILE
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.428007 16271 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:48.428761 16271 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[26887](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:42:48.434602 27819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:48.429223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36272 (local address 127.15.227.193:43649)
0812 14:42:48.429264 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:48.429265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:48.429275 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:48.429325 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:48.429325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:48.429332 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:48.429374 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:48.429658 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.430137 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.430740 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:48.430916 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:48.430965 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:48.432725 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:48.432727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:48.432727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:48.432733 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:48.433744 (+ 1011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.434005 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.434006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.434006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.434124 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:48.434268 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:48.434269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:48.434269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:48.434343 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:48.434402 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:48.434478 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:42:48.434529 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:42:48.435143 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 27085
I20260812 14:42:48.452203 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 27331
I20260812 14:42:48.464685 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 27587
I20260812 14:42:48.476159 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26907
2026-08-12T14:42:48Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3530 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[27840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[27840](info): set up 2 sockets
Aug 12 14:42:48 dist-test-slave-q9h9 krb5kdc[27840](info): commencing operation
krb5kdc: starting...
W20260812 14:42:49.679373 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:49 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:49Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab.
Entry for principal oryx/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab.
I20260812 14:42:49.792650 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36191
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab
--principal=oryx/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:36191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:49.872269 27866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:49.872455 27866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:49.872488 27866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:49.873567 27866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:49.873613 27866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:49.873632 27866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:49.873646 27866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:49.873663 27866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:49.875166 27866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:36191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab
--principal=oryx/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:49.875396 27866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:49.875602 27866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:49.880020 27878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:49.880038 27874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:49.880227 27866 server_base.cc:1034] running on GCE node
W20260812 14:42:49.880103 27883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:49.880472 27866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:49.880722 27866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:49.881862 27866 hybrid_clock.cc:648] HybridClock initialized: now 1786545769881847 us; error 28 us; skew 500 ppm
Aug 12 14:42:49 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545769, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:49.884063 27866 init.cc:377] Logged in from keytab as oryx/127.15.227.254@KRBTEST.COM (short username oryx)
I20260812 14:42:49.884433 27866 webserver.cc:460] Webserver started at http://127.15.227.254:44953/ using document root <none> and password file <none>
I20260812 14:42:49.884639 27866 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:49.884684 27866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:49.884812 27866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:49.885722 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "205e92ff322144cb919d11bfae3bdf89"
format_stamp: "Formatted at 2026-08-12 14:42:49 on dist-test-slave-q9h9"
server_key: "fdc47ea5053a2e162b08df588b8b8e09"
server_key_iv: "0f57038c99ee422f1a9419aad1d5a4dc"
server_key_version: "encryptionkey@0"
I20260812 14:42:49.886075 27866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "205e92ff322144cb919d11bfae3bdf89"
format_stamp: "Formatted at 2026-08-12 14:42:49 on dist-test-slave-q9h9"
server_key: "fdc47ea5053a2e162b08df588b8b8e09"
server_key_iv: "0f57038c99ee422f1a9419aad1d5a4dc"
server_key_version: "encryptionkey@0"
I20260812 14:42:49.887290 27866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:49.887882 27895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:49.888026 27866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:49.888104 27866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "205e92ff322144cb919d11bfae3bdf89"
format_stamp: "Formatted at 2026-08-12 14:42:49 on dist-test-slave-q9h9"
server_key: "fdc47ea5053a2e162b08df588b8b8e09"
server_key_iv: "0f57038c99ee422f1a9419aad1d5a4dc"
server_key_version: "encryptionkey@0"
I20260812 14:42:49.888164 27866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:49.902216 27866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:49.903910 27866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:49.904031 27866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:49.907816 27866 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:36191
I20260812 14:42:49.907850 28000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:36191 every 8 connection(s)
I20260812 14:42:49.908149 27866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:49.909049 28005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:49.911201 28005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: Bootstrap starting.
I20260812 14:42:49.911726 28005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:49.911949 28005 log.cc:824] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:49.912645 28005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: No bootstrap required, opened a new log
I20260812 14:42:49.914224 28005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } }
I20260812 14:42:49.914353 28005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:49.914381 28005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 205e92ff322144cb919d11bfae3bdf89, State: Initialized, Role: FOLLOWER
I20260812 14:42:49.914484 28005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } }
I20260812 14:42:49.914533 28005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:49.914552 28005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:49.914582 28005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:49.915305 28005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } }
I20260812 14:42:49.915392 28005 leader_election.cc:302] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 205e92ff322144cb919d11bfae3bdf89; no voters:
I20260812 14:42:49.915508 28005 leader_election.cc:288] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:49.915549 28009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:49.915699 28009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 LEADER]: Becoming Leader. State: Replica: 205e92ff322144cb919d11bfae3bdf89, State: Running, Role: LEADER
I20260812 14:42:49.915732 28005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:49.915786 28009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } }
I20260812 14:42:49.916117 28013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } }
I20260812 14:42:49.916132 28014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 205e92ff322144cb919d11bfae3bdf89. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } }
I20260812 14:42:49.916270 28013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:49.916664 28023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:49.916851 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 27866
I20260812 14:42:49.916954 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:49.917160 16271 external_mini_cluster.cc:1428] Setting key d7ee548f2f10043c0122f572a1a1a423
I20260812 14:42:49.917209 28023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:49.916678 28014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:49.921080 28023 catalog_manager.cc:1347] Generated new cluster ID: 7132973b1cd54dfa99e0f4a82d790bf2
I20260812 14:42:49.921146 28023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:42:49 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:49.927961 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:49.917577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.15.227.254:36191)
0812 14:42:49.917750 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:49.917759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:49.917997 (+ 238us) server_negotiation.cc:407] Connection header received
0812 14:42:49.918160 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:49.918168 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:49.918223 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:49.918354 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:49.919114 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:49.920340 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:49.921053 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:49.921680 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:49.921806 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:49.924486 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:49.924490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:49.924490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:49.924501 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:49.926364 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:49.926740 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:49.926742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:49.926743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:49.926834 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:49.927020 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:49.927021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:49.927021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:49.927279 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:49.927408 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:49.927505 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 14:42:49.927728 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260812 14:42:49.945904 28023 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:49.946409 28023 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:49.950198 28023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: Generated new TSK 0
I20260812 14:42:49.950346 28023 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab.
Entry for principal oryx/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab.
I20260812 14:42:49.976706 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36191
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab
--principal=oryx/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:50.057799 28055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:50.057981 28055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:50.058003 28055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:50.059123 28055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:50.059166 28055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:50.059191 28055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:50.060590 28055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab
--principal=oryx/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:50.060786 28055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:50.060994 28055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:50.065711 28065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.065829 28064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.065876 28055 server_base.cc:1034] running on GCE node
W20260812 14:42:50.065958 28068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.066157 28055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:50.066401 28055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:50.067538 28055 hybrid_clock.cc:648] HybridClock initialized: now 1786545770067525 us; error 35 us; skew 500 ppm
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:50.069839 28055 init.cc:377] Logged in from keytab as oryx/127.15.227.193@KRBTEST.COM (short username oryx)
I20260812 14:42:50.070276 28055 webserver.cc:460] Webserver started at http://127.15.227.193:43713/ using document root <none> and password file <none>
I20260812 14:42:50.070466 28055 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.070513 28055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.070628 28055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:50.071579 28055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "707d1c53fc2d42fd873b6f288c5ae04d"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "55af31194bf6739ee8fce3e7725a63e5"
server_key_iv: "c903e5e1240f3dd80b9c4634f9b021ef"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.071890 28055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "707d1c53fc2d42fd873b6f288c5ae04d"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "55af31194bf6739ee8fce3e7725a63e5"
server_key_iv: "c903e5e1240f3dd80b9c4634f9b021ef"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.073048 28055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:50.073714 28077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.073854 28055 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:50.073933 28055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "707d1c53fc2d42fd873b6f288c5ae04d"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "55af31194bf6739ee8fce3e7725a63e5"
server_key_iv: "c903e5e1240f3dd80b9c4634f9b021ef"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.073992 28055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.081126 28055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:50.082705 28055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:50.082821 28055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:50.083012 28055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:50.083277 28055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:50.083310 28055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.083335 28055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:50.083361 28055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.088780 28055 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:44449
I20260812 14:42:50.088840 28292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:44449 every 8 connection(s)
I20260812 14:42:50.089087 28055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:50.091368 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 28055
I20260812 14:42:50.091464 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:50.091673 16271 external_mini_cluster.cc:1428] Setting key 7f851b3361dc59b4c2d6c9cd587049cf
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:50.098489 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.089689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:57575 (local address 127.15.227.254:36191)
0812 14:42:50.089765 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.089768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.089915 (+ 147us) server_negotiation.cc:407] Connection header received
0812 14:42:50.090411 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.090412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.090426 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.090474 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.091415 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.092128 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.093130 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.093398 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.093462 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.095607 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.095608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.095609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.095616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.097114 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.097500 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.097502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.097503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.097568 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.097802 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.097802 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.097803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.097936 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.098003 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.098295 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.098398 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:42:50.098521 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.089641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.193:57575)
0812 14:42:50.089803 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.089836 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.090329 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.090531 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.090535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.091125 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.091328 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.091333 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.092219 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.092220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.093034 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.093046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.093477 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.093478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.093575 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.094084 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.094094 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.095516 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.097195 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.097202 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.097204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.097422 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.097635 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.097636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.097636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.097736 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.098007 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.098011 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.098168 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.098255 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.098374 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260812 14:42:50.099022 28296 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36191
I20260812 14:42:50.099124 28296 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:50.099305 28296 heartbeater.cc:511] Master 127.15.227.254:36191 requested a full tablet report, sending...
I20260812 14:42:50.099807 27934 ts_manager.cc:194] Registered new tserver with Master: 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449)
I20260812 14:42:50.100415 27934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:57575
WARNING: no policy specified for oryx/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab.
Entry for principal oryx/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab.
I20260812 14:42:50.116786 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36191
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab
--principal=oryx/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:50.195652 28301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:50.195843 28301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:50.195874 28301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:50.196954 28301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:50.196993 28301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:50.197027 28301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:50.198469 28301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab
--principal=oryx/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:50.198706 28301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:50.198904 28301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:50.203490 28313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.203495 28308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.203476 28309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.203632 28301 server_base.cc:1034] running on GCE node
I20260812 14:42:50.203898 28301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:50.204124 28301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:50.205259 28301 hybrid_clock.cc:648] HybridClock initialized: now 1786545770205254 us; error 36 us; skew 500 ppm
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:50.207683 28301 init.cc:377] Logged in from keytab as oryx/127.15.227.194@KRBTEST.COM (short username oryx)
I20260812 14:42:50.208060 28301 webserver.cc:460] Webserver started at http://127.15.227.194:36247/ using document root <none> and password file <none>
I20260812 14:42:50.208261 28301 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.208302 28301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.208391 28301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:50.209317 28301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "3b46588679eb406796517b8da42aaa34"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "f3538e78832ec8926bb19457e100f4b4"
server_key_iv: "90ea09c7db3897f5ae8aeaebc5ae908e"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.209599 28301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "3b46588679eb406796517b8da42aaa34"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "f3538e78832ec8926bb19457e100f4b4"
server_key_iv: "90ea09c7db3897f5ae8aeaebc5ae908e"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.210816 28301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:42:50.211483 28324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.211656 28301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:50.211748 28301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "3b46588679eb406796517b8da42aaa34"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "f3538e78832ec8926bb19457e100f4b4"
server_key_iv: "90ea09c7db3897f5ae8aeaebc5ae908e"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.211805 28301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.241974 28301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:50.244097 28301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:50.244262 28301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:50.244479 28301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:50.244779 28301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:50.244812 28301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.244841 28301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:50.244859 28301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.250174 28301 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:42803
I20260812 14:42:50.250208 28545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:42803 every 8 connection(s)
I20260812 14:42:50.250514 28301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:50.251060 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 28301
I20260812 14:42:50.251185 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:50.251405 16271 external_mini_cluster.cc:1428] Setting key d979a452a904e2b8419bbe7dcb2ade9e
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:50.259900 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.251232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:34135 (local address 127.15.227.254:36191)
0812 14:42:50.251286 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.251288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.251501 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:42:50.251952 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.251953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.251964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.252017 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.252932 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.253597 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.254893 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.255088 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.255142 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.257375 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.257376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.257376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.257383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.258615 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.259060 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.259060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.259061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.259105 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.259347 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.259348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.259348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.259439 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.259484 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.259777 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.259837 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:42:50.260133 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.251215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:34135)
0812 14:42:50.251371 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.251406 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.251876 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.252089 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.252095 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.252644 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.252835 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.252840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.253672 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.253673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.254813 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.254817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.255165 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.255166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.255303 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.255725 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.255732 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.257285 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.258695 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.258704 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.258706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.258993 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.259146 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.259146 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.259147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.259290 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.259494 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.259500 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.259653 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.259765 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.259922 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260812 14:42:50.260588 28546 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36191
I20260812 14:42:50.260670 28546 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:50.260825 28546 heartbeater.cc:511] Master 127.15.227.254:36191 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.15.227.195@KRBTEST.COM; defaulting to no policy
I20260812 14:42:50.261204 27934 ts_manager.cc:194] Registered new tserver with Master: 3b46588679eb406796517b8da42aaa34 (127.15.227.194:42803)
I20260812 14:42:50.261569 27934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:34135
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab.
Entry for principal oryx/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab.
I20260812 14:42:50.276049 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36191
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab
--principal=oryx/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:50.354395 28553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:50.354578 28553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:50.354609 28553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:50.355712 28553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:50.355752 28553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:50.355788 28553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:50.357223 28553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab
--principal=oryx/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:50.357450 28553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:50.357648 28553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:50.362294 28560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.362289 28568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.362288 28561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.362588 28553 server_base.cc:1034] running on GCE node
I20260812 14:42:50.362780 28553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:50.363011 28553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:50.364140 28553 hybrid_clock.cc:648] HybridClock initialized: now 1786545770364126 us; error 31 us; skew 500 ppm
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:50.366436 28553 init.cc:377] Logged in from keytab as oryx/127.15.227.195@KRBTEST.COM (short username oryx)
I20260812 14:42:50.366813 28553 webserver.cc:460] Webserver started at http://127.15.227.195:34449/ using document root <none> and password file <none>
I20260812 14:42:50.367012 28553 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.367059 28553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.367177 28553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:50.368091 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "4f770c70c2ff459abcd13c5e844a07be"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "6caafd40a300a6b517451b73013048b6"
server_key_iv: "8de3f3bd6d5324f2cc511734c871af87"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.368395 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "4f770c70c2ff459abcd13c5e844a07be"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "6caafd40a300a6b517451b73013048b6"
server_key_iv: "8de3f3bd6d5324f2cc511734c871af87"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.369524 28553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:42:50.370209 28579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.370353 28553 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.370430 28553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "4f770c70c2ff459abcd13c5e844a07be"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "6caafd40a300a6b517451b73013048b6"
server_key_iv: "8de3f3bd6d5324f2cc511734c871af87"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.370488 28553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.388594 28553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:50.390178 28553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:50.390290 28553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:50.390434 28553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:50.390653 28553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:50.390681 28553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.390705 28553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:50.390722 28553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.395973 28553 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:45467
I20260812 14:42:50.396005 28792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:45467 every 8 connection(s)
I20260812 14:42:50.396279 28553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:42:50.400226 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 28553
I20260812 14:42:50.400324 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:50.400482 16271 external_mini_cluster.cc:1428] Setting key 4680d76a892a8c9f3d6f31592b1a629c
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:50.404105 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.396892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:41003 (local address 127.15.227.254:36191)
0812 14:42:50.396938 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.396941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.397129 (+ 188us) server_negotiation.cc:407] Connection header received
0812 14:42:50.397539 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.397540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.397550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.397587 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.398455 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.398931 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.399801 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.399974 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.400013 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.402118 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.402119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.402119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.402126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.403131 (+ 1005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.403451 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.403452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.403452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.403496 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.403669 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.403669 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.403669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.403762 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.403804 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.403991 (+ 187us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.404045 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:42:50.404254 28799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.396872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.195:41003)
0812 14:42:50.397030 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.397061 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.397457 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.397645 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.397648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.398218 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.398387 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.398391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.398990 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.398990 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.399740 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.399743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.400022 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.400022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.400133 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.400639 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.400649 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.402048 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.403192 (+ 1144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.403198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.403199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.403397 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.403538 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.403538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.403539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.403630 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.403802 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.403804 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.403901 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.403971 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.404076 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260812 14:42:50.404652 28793 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36191
I20260812 14:42:50.404747 28793 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:50.404917 28793 heartbeater.cc:511] Master 127.15.227.254:36191 requested a full tablet report, sending...
I20260812 14:42:50.405213 27934 ts_manager.cc:194] Registered new tserver with Master: 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195:45467)
I20260812 14:42:50.405556 27934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:41003
I20260812 14:42:50.412019 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.227.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.253.keytab.
Entry for principal oryx/127.15.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.253.keytab.
I20260812 14:42:50.434701 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:42945
--webserver_interface=127.15.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.253.keytab
--principal=oryx/127.15.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:36191,127.15.227.253:42945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:50.512403 28806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:50.512583 28806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:50.512603 28806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:50.513679 28806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:50.513720 28806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:50.513733 28806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:50.513746 28806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:50.513761 28806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:50.515256 28806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:36191,127.15.227.253:42945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.253.keytab
--principal=oryx/127.15.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:42945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
W20260812 14:42:50.515368 28806 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.227.254:36191,127.15.227.253:42945'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 14:42:50.515532 28806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:50.515726 28806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:50.520283 28815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.520283 28820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.520417 28806 server_base.cc:1034] running on GCE node
W20260812 14:42:50.520573 28817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.520809 28806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:50.521090 28806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:50.522277 28806 hybrid_clock.cc:648] HybridClock initialized: now 1786545770522254 us; error 33 us; skew 500 ppm
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:50.524554 28806 init.cc:377] Logged in from keytab as oryx/127.15.227.253@KRBTEST.COM (short username oryx)
I20260812 14:42:50.524945 28806 webserver.cc:460] Webserver started at http://127.15.227.253:41727/ using document root <none> and password file <none>
I20260812 14:42:50.525156 28806 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.525209 28806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.525331 28806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:50.526333 28806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/instance:
uuid: "095f21efeb3e4a65a13ba79a4fe021c3"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "da3af14c2a26cb16c295bc4dac4fb46b"
server_key_iv: "90a856dfb8987260af1ab65c3567acdb"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.526645 28806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal/instance:
uuid: "095f21efeb3e4a65a13ba79a4fe021c3"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "da3af14c2a26cb16c295bc4dac4fb46b"
server_key_iv: "90a856dfb8987260af1ab65c3567acdb"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.527803 28806 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:50.528443 28829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.528592 28806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:50.528672 28806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
uuid: "095f21efeb3e4a65a13ba79a4fe021c3"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "da3af14c2a26cb16c295bc4dac4fb46b"
server_key_iv: "90a856dfb8987260af1ab65c3567acdb"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.528731 28806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.536543 28806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:50.538439 28806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:50.538558 28806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:50.542002 28806 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.253:42945
I20260812 14:42:50.542054 28929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.253:42945 every 8 connection(s)
I20260812 14:42:50.542361 28806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
I20260812 14:42:50.543253 28930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:50.544549 28930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:42:50.548595 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 28806
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.253@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:50.552518 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.545147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.15.227.254:36191)
0812 14:42:50.545200 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.545203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.545372 (+ 169us) server_negotiation.cc:407] Connection header received
0812 14:42:50.545783 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.545784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.545794 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.545841 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.546408 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.546884 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.547749 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.547917 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.547957 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.550004 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.550006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.550006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.550034 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.551266 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.551652 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.551653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.551654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.551713 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.551925 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.551926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.551927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.552121 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.552199 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.552395 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.552457 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:42:50.552639 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.545112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53068)
0812 14:42:50.545271 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.545303 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.545713 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.545889 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.545892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.546159 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.546337 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.546341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.546942 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.546942 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.547688 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.547690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.547981 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.547982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.548082 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.548467 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.548475 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.549924 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.551345 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.551352 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.551354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.551585 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.551769 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.551770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.551770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.551871 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.552180 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.552185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.552298 (+ 113us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.552377 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.552483 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260812 14:42:50.553236 28930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.227.253@KRBTEST.COM
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.253@KRBTEST.COM for oryx/127.15.227.253@KRBTEST.COM
I20260812 14:42:50.557058 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.549118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.15.227.253:42945)
0812 14:42:50.549256 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.549260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.549328 (+ 68us) server_negotiation.cc:407] Connection header received
0812 14:42:50.549352 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.549354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.549375 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.549459 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.550039 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.550670 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.551413 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.551681 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.551735 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.554641 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.554648 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.554648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.554655 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.556033 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.556338 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.556341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.556341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.556389 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.556582 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.556583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.556583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.556700 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.556794 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.556908 (+ 114us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.556968 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260812 14:42:50.559813 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.553456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.0.0.1:34314)
0812 14:42:50.553617 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.553627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.553675 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.553779 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.553780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.553905 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.553948 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.553949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.554673 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.554674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.555280 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.555283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.555723 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.555724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.555773 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.556064 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.556068 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.557211 (+ 1143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.558987 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.558988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.558988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.559130 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.559291 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.559292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.559292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.559343 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.559530 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.559531 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.559565 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.559623 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.559709 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260812 14:42:50.559894 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.553515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34314 (local address 127.15.227.253:42945)
0812 14:42:50.553637 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.553639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.553662 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:42:50.553715 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.553716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.553724 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.553779 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.554008 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.554601 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.555359 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.555654 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.555724 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.557314 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.557314 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.557315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.557320 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.558921 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.559184 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.559185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.559185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.559232 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.559390 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.559391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.559391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.559482 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.559534 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.559647 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.559758 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260812 14:42:50.560835 28930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Bootstrap starting.
I20260812 14:42:50.561333 28930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:50.561540 28930 log.cc:824] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:50.562179 28930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: No bootstrap required, opened a new log
I20260812 14:42:50.563418 28930 raft_consensus.cc:348] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } }
I20260812 14:42:50.563520 28930 raft_consensus.cc:374] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:50.563541 28930 raft_consensus.cc:729] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095f21efeb3e4a65a13ba79a4fe021c3, State: Initialized, Role: FOLLOWER
I20260812 14:42:50.563630 28930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } }
I20260812 14:42:50.563869 28930 sys_catalog.cc:564] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:50.563853 28945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } } }
I20260812 14:42:50.563971 28945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:42:50.565248 28970 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:42:50.565297 28970 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:42:50.566860 28806 master_runner.cc:428] Detected that this master 095f21efeb3e4a65a13ba79a4fe021c3 is joining an existing cluster
I20260812 14:42:50.566933 28806 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.227.253:42945
I20260812 14:42:50.573738 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.567835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34322 (local address 127.15.227.253:42945)
0812 14:42:50.567867 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.567869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.567988 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:42:50.568075 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.568076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.568084 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.568108 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.568519 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.569202 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.569882 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.570157 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.570208 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.570964 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.570965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.570966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.570972 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.572261 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.572581 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.572582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.572582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.572637 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.573218 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.573219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.573220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.573360 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.573425 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.573590 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.573662 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:42:50.574203 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.567813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.0.0.1:34322)
0812 14:42:50.567930 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.567939 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.567994 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.568153 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.568153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.568373 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.568443 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.568445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.569289 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.569289 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.569813 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.569814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.570241 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.570243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.570304 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.570627 (+ 323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.570632 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.570890 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.572327 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.572328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.572328 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.572514 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.572756 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.572757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.572758 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.573079 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.573448 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.573448 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.573488 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.573940 (+ 452us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.574106 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260812 14:42:50.574982 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.567965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53072 (local address 127.15.227.254:36191)
0812 14:42:50.568022 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.568024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.568066 (+ 42us) server_negotiation.cc:407] Connection header received
0812 14:42:50.568120 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.568121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.568130 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.568186 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.569320 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.570065 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.571073 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.571373 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.571410 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.572419 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.572420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.572420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.572427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.573690 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.574203 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.574204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.574205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.574261 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.574477 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.574479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.574479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.574605 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.574649 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.574863 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.574924 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:42:50.575007 28985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.567855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53072)
0812 14:42:50.568019 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.568028 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.568075 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.568196 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.568196 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.568400 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.568445 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.568446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.570150 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.570151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.570975 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.570978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.571481 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.571482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.571559 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.571874 (+ 315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.571878 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.572344 (+ 466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.573772 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.573773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.573773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.574135 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.574333 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.574334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.574334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.574395 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.574696 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.574697 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.574736 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.574824 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.574914 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":62,"threads_started":1}
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545769, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.254@KRBTEST.COM for oryx/127.15.227.253@KRBTEST.COM
I20260812 14:42:50.583555 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.577004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34328 (local address 127.15.227.253:42945)
0812 14:42:50.577040 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.577042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.577161 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:42:50.577279 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.577279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.577287 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.577327 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.577753 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.578256 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.579296 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.579459 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.579502 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.581435 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.581436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.581436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.581442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.582616 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.582994 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.582994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.582994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.583036 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.583194 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.583195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.583195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.583272 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.583312 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.583463 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.583508 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:42:50.583664 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.576944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.0.0.1:34328)
0812 14:42:50.577044 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.577080 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.577197 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.577332 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.577335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.577641 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.577691 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.577693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.578333 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.578334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.579219 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.579221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.579515 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.579516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.579587 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.579959 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.579964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.581344 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.582681 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.582687 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.582689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.582930 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.583084 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.583085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.583085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.583145 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.583318 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.583323 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.583370 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.583449 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.583555 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":51,"threads_started":1}
I20260812 14:42:50.584198 27934 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.227.253:42945: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3"
member_type: NON_VOTER
last_known_addr {
host: "127.15.227.253"
port: 42945
}
attrs {
promote: true
}
}
dest_uuid: "205e92ff322144cb919d11bfae3bdf89"
cas_config_opid_index: -1
I20260812 14:42:50.584537 27934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } }
I20260812 14:42:50.584997 28988 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 095f21efeb3e4a65a13ba79a4fe021c3 (127.15.227.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } }
I20260812 14:42:50.585227 28885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:50.585886 28885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Refusing update from remote peer 205e92ff322144cb919d11bfae3bdf89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 14:42:50.585924 28988 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.227.253:42945
I20260812 14:42:50.586239 28806 master.cc:537] Master@127.15.227.253:42945 shutting down...
I20260812 14:42:50.586252 28993 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:50.586710 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } }
I20260812 14:42:50.586791 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
W20260812 14:42:50.586907 27896 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 14:42:50.587301 27896 consensus_peers.cc:487] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 -> Peer 095f21efeb3e4a65a13ba79a4fe021c3 (127.15.227.253:42945): Couldn't send request to peer 095f21efeb3e4a65a13ba79a4fe021c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 14:42:50.588933 28806 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:50.589015 28806 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:50.589062 28806 tablet_replica.cc:330] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: stopping tablet replica
I20260812 14:42:50.601341 28806 master.cc:552] Master@127.15.227.253:42945 shutdown complete.
I20260812 14:42:50.601388 28806 master_runner.cc:305] Clearing existing system tablet
I20260812 14:42:50.601507 28806 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.601552 28806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.602216 28806 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:50.602700 28998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.602809 28806 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.602865 28806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.602890 28806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
uuid: "095f21efeb3e4a65a13ba79a4fe021c3"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "da3af14c2a26cb16c295bc4dac4fb46b"
server_key_iv: "90a856dfb8987260af1ab65c3567acdb"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.603709 28806 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:50.605015 28806 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 14:42:50.605082 28806 log.cc:1196] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 14:42:50.605283 28806 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Deleting consensus metadata
I20260812 14:42:50.605470 28806 master_runner.cc:315] Copying system tablet from 127.15.227.254:36191
I20260812 14:42:50.606053 28806 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: tablet copy: Beginning tablet copy session from remote peer at address 127.15.227.254:36191
I20260812 14:42:50.611701 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.606362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53074 (local address 127.15.227.254:36191)
0812 14:42:50.606409 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.606411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.606469 (+ 58us) server_negotiation.cc:407] Connection header received
0812 14:42:50.606535 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.606536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.606544 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.606587 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.606888 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.607350 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.608346 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.608531 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.608588 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.609463 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.609464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.609464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.609470 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.610796 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.611136 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.611137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.611137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.611182 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.611347 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.611347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.611347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.611428 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.611470 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.611598 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.611646 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:42:50.611778 29007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.606304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53074)
0812 14:42:50.606412 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.606422 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.606482 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.606592 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.606593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.606769 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.606830 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.606832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.607409 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.607410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.608254 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.608257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.608589 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.608590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.608657 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.609014 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.609019 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.609382 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.610864 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.610865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.610865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.611072 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.611230 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.611231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.611231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.611299 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.611473 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.611474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.611515 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.611582 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.611674 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":67,"threads_started":1}
I20260812 14:42:50.612089 27973 tablet_copy_service.cc:140] P 205e92ff322144cb919d11bfae3bdf89: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 095f21efeb3e4a65a13ba79a4fe021c3 ({username='oryx', principal='oryx/127.15.227.253@KRBTEST.COM'} at 127.0.0.1:53074)
I20260812 14:42:50.612201 27973 tablet_copy_service.cc:161] P 205e92ff322144cb919d11bfae3bdf89: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 095f21efeb3e4a65a13ba79a4fe021c3 at {username='oryx', principal='oryx/127.15.227.253@KRBTEST.COM'} at 127.0.0.1:53074: session id = 095f21efeb3e4a65a13ba79a4fe021c3-00000000000000000000000000000000
I20260812 14:42:50.612830 27973 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 14:42:50.613366 28806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 14:42:50.614495 28806 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: tablet copy: Starting download of 0 data blocks...
I20260812 14:42:50.614631 28806 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: tablet copy: Starting download of 1 WAL segments...
I20260812 14:42:50.615497 28806 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 14:42:50.616339 27973 tablet_copy_service.cc:342] P 205e92ff322144cb919d11bfae3bdf89: Request end of tablet copy session 095f21efeb3e4a65a13ba79a4fe021c3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.227.253@KRBTEST.COM'} at 127.0.0.1:53074
I20260812 14:42:50.616384 27973 tablet_copy_service.cc:433] P 205e92ff322144cb919d11bfae3bdf89: ending tablet copy session 095f21efeb3e4a65a13ba79a4fe021c3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 095f21efeb3e4a65a13ba79a4fe021c3
I20260812 14:42:50.618526 28806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:50.619499 29009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.619604 29008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.619719 29014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.619820 28806 server_base.cc:1034] running on GCE node
I20260812 14:42:50.619900 28806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:50.620018 28806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:50.621132 28806 hybrid_clock.cc:648] HybridClock initialized: now 1786545770621120 us; error 24 us; skew 500 ppm
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:50.622298 28806 init.cc:377] Logged in from keytab as oryx/127.15.227.253@KRBTEST.COM (short username oryx)
I20260812 14:42:50.622531 28806 webserver.cc:460] Webserver started at http://127.15.227.253:40885/ using document root <none> and password file <none>
I20260812 14:42:50.622653 28806 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.622685 28806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.623217 28806 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:50.623693 29028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.623805 28806 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.623862 28806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
uuid: "095f21efeb3e4a65a13ba79a4fe021c3"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "da3af14c2a26cb16c295bc4dac4fb46b"
server_key_iv: "90a856dfb8987260af1ab65c3567acdb"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.623909 28806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.637588 28806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:50.639083 28806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:50.641975 28806 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.253:42945
I20260812 14:42:50.642300 29114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.253:42945 every 8 connection(s)
I20260812 14:42:50.642376 28806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
I20260812 14:42:50.643088 29117 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:42:50.643474 29117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Bootstrap starting.
I20260812 14:42:50.645433 29117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:42:50.645627 29117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: Bootstrap complete.
I20260812 14:42:50.645789 29117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } }
I20260812 14:42:50.645848 29117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 095f21efeb3e4a65a13ba79a4fe021c3, State: Initialized, Role: LEARNER
I20260812 14:42:50.645872 29117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } }
I20260812 14:42:50.646056 29117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:50.646039 29121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } }
I20260812 14:42:50.646162 29121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:42:50.647351 29140 catalog_manager.cc:1259] Loaded cluster ID: 7132973b1cd54dfa99e0f4a82d790bf2
I20260812 14:42:50.647393 29140 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: loading cluster ID for follower catalog manager: success
I20260812 14:42:50.648213 29140 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: acquiring CA information for follower catalog manager: success
I20260812 14:42:50.648454 29140 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:42:50.664623 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.658346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.15.227.253:42945)
0812 14:42:50.658448 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.658449 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.658456 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:50.658474 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.658475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.658483 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.658542 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.659176 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.659792 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.660809 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.660986 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.661027 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.662324 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.662325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.662325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.662331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.663633 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.663997 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.663997 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.663998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.664041 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.664222 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.664223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.664223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.664311 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.664374 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.664500 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.664553 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":55,"threads_started":1}
WARNING: no policy specified for oryx/127.15.227.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.252.keytab.
Entry for principal oryx/127.15.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.252.keytab.
I20260812 14:42:50.686761 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:42215
--webserver_interface=127.15.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.252.keytab
--principal=oryx/127.15.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:50.765985 29145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:50.766183 29145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:50.766204 29145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:50.767336 29145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:50.767374 29145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:50.767386 29145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:50.767397 29145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:50.767422 29145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:50.768884 29145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.252.keytab
--principal=oryx/127.15.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:42215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:50.769090 29145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:50.769294 29145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:50.773785 29153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.773839 29158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.773797 29161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.773869 29145 server_base.cc:1034] running on GCE node
I20260812 14:42:50.774200 29145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:50.774425 29145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:50.775560 29145 hybrid_clock.cc:648] HybridClock initialized: now 1786545770775531 us; error 40 us; skew 500 ppm
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:50.777880 29145 init.cc:377] Logged in from keytab as oryx/127.15.227.252@KRBTEST.COM (short username oryx)
I20260812 14:42:50.778298 29145 webserver.cc:460] Webserver started at http://127.15.227.252:38293/ using document root <none> and password file <none>
I20260812 14:42:50.778481 29145 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.778522 29145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.778621 29145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:50.779565 29145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/instance:
uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "659bc10af8071c7ab3337c8102722c7e"
server_key_iv: "f1492c2093f37358c2ee2adf99ab2786"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.779873 29145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal/instance:
uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "659bc10af8071c7ab3337c8102722c7e"
server_key_iv: "f1492c2093f37358c2ee2adf99ab2786"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.781041 29145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:42:50.781679 29172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.781811 29145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:50.781891 29145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "659bc10af8071c7ab3337c8102722c7e"
server_key_iv: "f1492c2093f37358c2ee2adf99ab2786"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.781947 29145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.793401 29145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:50.795001 29145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:50.795120 29145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:50.798765 29145 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.252:42215
I20260812 14:42:50.798795 29263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.252:42215 every 8 connection(s)
I20260812 14:42:50.799079 29145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
I20260812 14:42:50.799988 29265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:50.800995 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 29145
I20260812 14:42:50.801527 29265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.252@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.227.252@KRBTEST.COM
I20260812 14:42:50.809613 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.802346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53078 (local address 127.15.227.254:36191)
0812 14:42:50.802404 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.802406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.802456 (+ 50us) server_negotiation.cc:407] Connection header received
0812 14:42:50.802527 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.802528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.802539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.802586 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.803086 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.803574 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.804407 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.804614 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.804668 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.806874 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.806875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.806876 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.806883 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.808339 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.808740 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.808741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.808742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.808805 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.809023 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.809023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.809024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.809145 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.809201 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.809444 (+ 243us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.809524 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:42:50.809876 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.802159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53078)
0812 14:42:50.802282 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.802395 (+ 113us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.802476 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.802603 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.802606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.802844 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.802989 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.802993 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.803660 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.803661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.804317 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.804319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.804700 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.804701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.804800 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.805219 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.805230 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.806792 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.808408 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.808414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.808416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.808676 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.808864 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.808865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.808865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.808965 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.809200 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.809206 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.809328 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.809412 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.809546 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260812 14:42:50.809880 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.801665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40346 (local address 127.15.227.252:42215)
0812 14:42:50.801841 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.801849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.801879 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:42:50.802358 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.802367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.802417 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.802522 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.803070 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.803834 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.804509 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.804819 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.804879 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.807341 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.807349 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.807349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.807355 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.808787 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.809091 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.809094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.809094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.809142 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.809312 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.809312 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.809313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.809431 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.809549 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.809631 (+ 82us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.809803 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260812 14:42:50.810395 29265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.252@KRBTEST.COM for oryx/127.15.227.253@KRBTEST.COM
I20260812 14:42:50.815766 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.810610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.0.0.1:34370)
0812 14:42:50.810640 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.810649 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.810683 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.810800 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.810800 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.810898 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.810930 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.810931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.811485 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.811485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.811935 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.811936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.812231 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.812232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.812288 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.812659 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.812664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.813717 (+ 1053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.814993 (+ 1276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.814994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.814995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.815209 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.815365 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.815366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.815366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.815417 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.815571 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.815572 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.815602 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.815651 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.815713 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 14:42:50.815791 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.810664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.15.227.253:42945)
0812 14:42:50.810701 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.810703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.810724 (+ 21us) server_negotiation.cc:407] Connection header received
0812 14:42:50.810748 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.810749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.810759 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.810792 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.810976 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.811435 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.811984 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.812159 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.812202 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.813798 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.813799 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.813799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.813805 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.814929 (+ 1124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.815268 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.815269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.815270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.815321 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.815450 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.815450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.815450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.815534 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.815575 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.815667 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.815732 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:42:50.816138 29265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.252@KRBTEST.COM for oryx/127.15.227.252@KRBTEST.COM
I20260812 14:42:50.821158 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.816389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40354 (local address 127.15.227.252:42215)
0812 14:42:50.816417 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.816418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.816471 (+ 53us) server_negotiation.cc:407] Connection header received
0812 14:42:50.816509 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.816509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.816516 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.816553 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.816733 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.817142 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.817704 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.817877 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.817917 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.819257 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.819258 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.819258 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.819264 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.820201 (+ 937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.820573 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.820574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.820574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.820625 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.820783 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.820783 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.820784 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.820867 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.820911 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.821033 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.821091 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 14:42:50.821301 29274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.816344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.0.0.1:40354)
0812 14:42:50.816418 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.816426 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.816475 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.816557 (+ 82us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.816558 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.816665 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.816691 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.816692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.817189 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.817190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.817651 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.817652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.817921 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.817922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.817953 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.818252 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.818255 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.819183 (+ 928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.820319 (+ 1136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.820320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.820320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.820499 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.820678 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.820679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.820679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.820725 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.820921 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.820922 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.820957 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.821015 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.821139 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":34}
I20260812 14:42:50.822531 29265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Bootstrap starting.
I20260812 14:42:50.823354 29265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:50.823616 29265 log.cc:824] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:50.824329 29265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: No bootstrap required, opened a new log
I20260812 14:42:50.825727 29265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } }
I20260812 14:42:50.825855 29265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:50.825892 29265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ebdf2c2baf74bfcba8af5bf8abef14f, State: Initialized, Role: FOLLOWER
I20260812 14:42:50.826048 29265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } }
I20260812 14:42:50.826311 29277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } } }
I20260812 14:42:50.826357 29265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:50.826391 29277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:42:50.827860 29297 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:42:50.827919 29297 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:42:50.829975 29145 master_runner.cc:428] Detected that this master 6ebdf2c2baf74bfcba8af5bf8abef14f is joining an existing cluster
I20260812 14:42:50.830035 29145 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.227.252:42215
I20260812 14:42:50.837232 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.831147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.0.0.1:40366)
0812 14:42:50.831278 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.831287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.831343 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.831480 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.831480 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.831794 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.831892 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.831894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.832737 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.832738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.833615 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.833618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.834121 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.834122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.834176 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.834546 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.834551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.834859 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.836417 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.836418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.836418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.836568 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.836761 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.836761 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.836762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.836805 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.837022 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.837022 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.837051 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.837093 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.837163 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":34,"thread_start_us":68,"threads_started":1}
I20260812 14:42:50.837332 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.831162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.15.227.252:42215)
0812 14:42:50.831193 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.831195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.831324 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:42:50.831402 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.831403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.831409 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.831433 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.831974 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.832649 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.833700 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.833979 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.834045 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.834948 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.834949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.834949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.834956 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.836339 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.836634 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.836635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.836635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.836696 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.836845 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.836845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.836846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.836965 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.837043 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.837119 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.837228 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":16}
I20260812 14:42:50.857198 29309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.831067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.0.0.1:34374)
0812 14:42:50.831259 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.831268 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.831343 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.831499 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.831499 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.831793 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.831901 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.831903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.832762 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.832762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.833621 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.833622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.834130 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.834131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.834183 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.854397 (+ 20214us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.854402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.854776 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.856278 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.856279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.856280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.856434 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.856602 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.856602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.856603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.856658 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.856876 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.856877 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.856917 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.856985 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.857096 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":59,"thread_start_us":85,"threads_started":1}
I20260812 14:42:50.857234 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.830987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.15.227.253:42945)
0812 14:42:50.831059 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.831060 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.831319 (+ 259us) server_negotiation.cc:407] Connection header received
0812 14:42:50.831404 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.831406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.831415 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.831442 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.831979 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.832684 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.833706 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.833992 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.834066 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.854872 (+ 20806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.854873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.854873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.854880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.856204 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.856493 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.856494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.856494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.856549 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.856710 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.856710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.856711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.856822 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.856875 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.857047 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.857136 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 14:42:50.858039 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.830926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53086)
0812 14:42:50.831221 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.831231 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.831317 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.831505 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.831505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.831803 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.831932 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.831934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.832845 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.832846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.833655 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.833658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.834219 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.834220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.834283 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.854541 (+ 20258us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.854546 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.854977 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.856902 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.856903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.856903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.857131 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.857356 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.857357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.857358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.857414 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.857709 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.857709 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.857758 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.857824 (+ 66us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.857918 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":57,"thread_start_us":80,"threads_started":1}
I20260812 14:42:50.858076 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.830971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.15.227.254:36191)
0812 14:42:50.831018 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.831020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.831289 (+ 269us) server_negotiation.cc:407] Connection header received
0812 14:42:50.831394 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.831396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.831407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.831442 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.832053 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.832738 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.833765 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.834090 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.834138 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.855108 (+ 20970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.855110 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.855110 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.855118 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.856640 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.857215 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.857216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.857216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.857279 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.857482 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.857484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.857484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.857630 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.857716 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.857859 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.857941 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:42:50.861174 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 28055
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545769, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.254@KRBTEST.COM for oryx/127.15.227.252@KRBTEST.COM
I20260812 14:42:50.888911 29271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.859895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.15.227.252:42215)
0812 14:42:50.859927 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.859929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.859940 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:50.860018 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.860019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.860026 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.860066 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.860347 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.860806 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.884441 (+ 23635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.884643 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.884722 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.886682 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.886684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.886685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.886693 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.887884 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.888201 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.888202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.888203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.888255 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.888417 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.888418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.888418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.888524 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.888580 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.888745 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.888815 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:42:50.888912 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.859852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.0.0.1:40378)
0812 14:42:50.859906 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.859915 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.859951 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.860074 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.860075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.860238 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.860284 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.860286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.860907 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.860909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.861608 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.861611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.884753 (+ 23142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.884755 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.884838 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.885294 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.885299 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.886593 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.887967 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.887969 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.887970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.888139 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.888310 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.888311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.888311 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.888367 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.888581 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.888582 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.888632 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.888693 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.888812 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:42:50.889405 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:44449
--local_ip_for_outbound_sockets=127.15.227.193
--tserver_master_addrs=127.15.227.254:36191
--webserver_port=43713
--webserver_interface=127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab
--principal=oryx/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:42:50.891052 27934 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.227.252:42215: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f"
member_type: NON_VOTER
last_known_addr {
host: "127.15.227.252"
port: 42215
}
attrs {
promote: true
}
}
dest_uuid: "205e92ff322144cb919d11bfae3bdf89"
cas_config_opid_index: 5
I20260812 14:42:50.891330 27934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } }
I20260812 14:42:50.891851 28993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.892879 28993 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.227.252:42215
I20260812 14:42:50.892962 28993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.893049 28993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:50.893314 29145 master.cc:537] Master@127.15.227.252:42215 shutting down...
I20260812 14:42:50.893419 29215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:50.894245 29215 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Refusing update from remote peer 205e92ff322144cb919d11bfae3bdf89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 14:42:50.895231 28987 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 14:42:50.895884 27898 consensus_peers.cc:487] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 -> Peer 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252:42215): Couldn't send request to peer 6ebdf2c2baf74bfcba8af5bf8abef14f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260812 14:42:50.897009 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.891800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.0.0.1:34376)
0812 14:42:50.893292 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.893302 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.893342 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.893565 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.893566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.893568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:50.893612 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.893613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.894615 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.894616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.895921 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.895925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.896756 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.896758 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.896826 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.896852 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.896903 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.896943 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1477}
I20260812 14:42:50.897269 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.891857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.15.227.253:42945)
0812 14:42:50.893260 (+ 1403us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.893262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.893396 (+ 134us) server_negotiation.cc:407] Connection header received
0812 14:42:50.893425 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.893426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.893449 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.893507 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:50.893687 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.894453 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.896006 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.896568 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.896640 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.896719 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.897148 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.897194 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1373}
I20260812 14:42:50.897287 29145 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:50.897368 29145 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:50.897410 29145 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: stopping tablet replica
I20260812 14:42:50.897452 29071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 LEARNER]: Refusing update from remote peer 205e92ff322144cb919d11bfae3bdf89: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 14:42:50.897625 28987 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 14:42:50.898217 29071 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.899112 29121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 205e92ff322144cb919d11bfae3bdf89. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.899188 29121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:42:50.899220 29121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: NON_VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: true } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.899276 29121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: LEARNER
I20260812 14:42:50.899595 28987 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: attempting to promote NON_VOTER 095f21efeb3e4a65a13ba79a4fe021c3 to VOTER
I20260812 14:42:50.899786 28987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } }
I20260812 14:42:50.900763 29071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 LEARNER]: Refusing update from remote peer 205e92ff322144cb919d11bfae3bdf89: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 14:42:50.900946 28988 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 14:42:50.901407 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.900542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.0.0.1:40388)
0812 14:42:50.901334 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.901377 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:42215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":67,"threads_started":1}
W20260812 14:42:50.901461 27898 consensus_peers.cc:487] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 -> Peer 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252:42215): Couldn't send request to peer 6ebdf2c2baf74bfcba8af5bf8abef14f. Status: Network error: Client connection negotiation failed: client connection to 127.15.227.252:42215: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 14:42:50.901640 28987 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 095f21efeb3e4a65a13ba79a4fe021c3 (127.15.227.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.901798 29071 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 095f21efeb3e4a65a13ba79a4fe021c3 (127.15.227.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.902559 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.902630 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:50.902752 28993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.902809 28993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:50.902808 29318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.902875 29318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:42:50.911972 29145 master.cc:552] Master@127.15.227.252:42215 shutdown complete.
I20260812 14:42:50.912014 29145 master_runner.cc:305] Clearing existing system tablet
I20260812 14:42:50.912134 29145 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.912176 29145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.912727 29145 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:50.913159 29324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.913275 29145 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.913326 29145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.913348 29145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "659bc10af8071c7ab3337c8102722c7e"
server_key_iv: "f1492c2093f37358c2ee2adf99ab2786"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.913923 29145 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:50.915081 29145 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 14:42:50.915143 29145 log.cc:1196] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 14:42:50.915334 29145 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Deleting consensus metadata
I20260812 14:42:50.915488 29145 master_runner.cc:315] Copying system tablet from 127.15.227.254:36191
I20260812 14:42:50.916093 29145 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.227.254:36191
I20260812 14:42:50.923401 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.916402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53094 (local address 127.15.227.254:36191)
0812 14:42:50.916482 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:50.916484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:50.918365 (+ 1881us) server_negotiation.cc:407] Connection header received
0812 14:42:50.918422 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:50.918423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:50.918433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:50.918479 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:50.918770 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.919255 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.919920 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.920107 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.920160 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.921007 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:50.921008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:50.921009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:50.921016 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:50.922393 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.922702 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.922703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.922704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.922768 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:50.922932 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:50.922932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:50.922933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:50.923040 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:50.923096 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:50.923243 (+ 147us) server_negotiation.cc:299] Negotiation successful
0812 14:42:50.923322 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:42:50.923635 29334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:50.916356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53094)
0812 14:42:50.918298 (+ 1942us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:50.918308 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:50.918365 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:50.918482 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:50.918483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:50.918656 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:50.918697 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.918699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.919326 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.919327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.919846 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:50.919848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:50.920168 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:50.920169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:50.920230 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:50.920631 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:50.920636 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:50.920923 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:50.922468 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.922469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.922469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.922638 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.922821 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:50.922822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:50.922822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:50.922882 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:50.923094 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:50.923094 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:50.923153 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 14:42:50.923213 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:42:50.923322 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1919,"thread_start_us":67,"threads_started":1}
I20260812 14:42:50.923974 27973 tablet_copy_service.cc:140] P 205e92ff322144cb919d11bfae3bdf89: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6ebdf2c2baf74bfcba8af5bf8abef14f ({username='oryx', principal='oryx/127.15.227.252@KRBTEST.COM'} at 127.0.0.1:53094)
I20260812 14:42:50.924031 27973 tablet_copy_service.cc:161] P 205e92ff322144cb919d11bfae3bdf89: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6ebdf2c2baf74bfcba8af5bf8abef14f at {username='oryx', principal='oryx/127.15.227.252@KRBTEST.COM'} at 127.0.0.1:53094: session id = 6ebdf2c2baf74bfcba8af5bf8abef14f-00000000000000000000000000000000
I20260812 14:42:50.924489 27973 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 14:42:50.924909 29145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 14:42:50.925823 29145 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: tablet copy: Starting download of 0 data blocks...
I20260812 14:42:50.926041 29145 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: tablet copy: Starting download of 1 WAL segments...
I20260812 14:42:50.926689 29145 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 14:42:50.927493 27973 tablet_copy_service.cc:342] P 205e92ff322144cb919d11bfae3bdf89: Request end of tablet copy session 6ebdf2c2baf74bfcba8af5bf8abef14f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.227.252@KRBTEST.COM'} at 127.0.0.1:53094
I20260812 14:42:50.927544 27973 tablet_copy_service.cc:433] P 205e92ff322144cb919d11bfae3bdf89: ending tablet copy session 6ebdf2c2baf74bfcba8af5bf8abef14f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6ebdf2c2baf74bfcba8af5bf8abef14f
I20260812 14:42:50.930243 29145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:50.931258 29335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:50.934854 29336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.935111 29145 server_base.cc:1034] running on GCE node
W20260812 14:42:50.935207 29338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:50.935401 29145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:50.935525 29145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:50.936635 29145 hybrid_clock.cc:648] HybridClock initialized: now 1786545770936628 us; error 24 us; skew 500 ppm
Aug 12 14:42:50 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545770, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:50.937780 29145 init.cc:377] Logged in from keytab as oryx/127.15.227.252@KRBTEST.COM (short username oryx)
I20260812 14:42:50.938031 29145 webserver.cc:460] Webserver started at http://127.15.227.252:37057/ using document root <none> and password file <none>
I20260812 14:42:50.938144 29145 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:50.938177 29145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:50.938768 29145 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:50.939234 29349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.939333 29145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:50.939368 29145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "659bc10af8071c7ab3337c8102722c7e"
server_key_iv: "f1492c2093f37358c2ee2adf99ab2786"
server_key_version: "encryptionkey@0"
I20260812 14:42:50.939420 29145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:50.949015 29145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:50.950503 29145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:50.953255 29145 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.252:42215
I20260812 14:42:50.953637 29145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
I20260812 14:42:50.954358 29436 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:42:50.954746 29436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Bootstrap starting.
I20260812 14:42:50.956943 29436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:42:50.957129 29436 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Bootstrap complete.
I20260812 14:42:50.957325 29436 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } }
I20260812 14:42:50.957394 29436 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6ebdf2c2baf74bfcba8af5bf8abef14f, State: Initialized, Role: LEARNER
I20260812 14:42:50.957430 29436 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } }
I20260812 14:42:50.957598 29436 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:50.957593 29440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:50.957821 29440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: This master's current role is: LEARNER
I20260812 14:42:50.958935 29462 catalog_manager.cc:1259] Loaded cluster ID: 7132973b1cd54dfa99e0f4a82d790bf2
I20260812 14:42:50.959657 29462 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: loading cluster ID for follower catalog manager: success
I20260812 14:42:50.960443 29462 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: acquiring CA information for follower catalog manager: success
I20260812 14:42:50.960681 29462 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:42:50.966107 29435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.252:42215 every 8 connection(s)
W20260812 14:42:51.013303 29310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:51.013581 29310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:51.013619 29310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:51.015270 29310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:51.015328 29310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:51.015357 29310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:51.017161 29310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab
--principal=oryx/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:44449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=43713
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:51.017378 29310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:51.017590 29310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:51.022862 29475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:51.022871 29470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:51.022884 29471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:51.023141 29310 server_base.cc:1034] running on GCE node
I20260812 14:42:51.023383 29310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:51.023622 29310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:51.024842 29310 hybrid_clock.cc:648] HybridClock initialized: now 1786545771024810 us; error 41 us; skew 500 ppm
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:51.027657 29310 init.cc:377] Logged in from keytab as oryx/127.15.227.193@KRBTEST.COM (short username oryx)
I20260812 14:42:51.028067 29310 webserver.cc:460] Webserver started at http://127.15.227.193:43713/ using document root <none> and password file <none>
I20260812 14:42:51.028247 29310 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:51.028299 29310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:51.029744 29310 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:51.030441 29487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.030601 29310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:42:51.030694 29310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "707d1c53fc2d42fd873b6f288c5ae04d"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "55af31194bf6739ee8fce3e7725a63e5"
server_key_iv: "c903e5e1240f3dd80b9c4634f9b021ef"
server_key_version: "encryptionkey@0"
I20260812 14:42:51.031035 29310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:51.050835 29310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:51.052457 29310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:51.052561 29310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:51.052726 29310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:51.053017 29310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:51.053051 29310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.053071 29310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:51.053102 29310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.058650 29310 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:44449
I20260812 14:42:51.058784 29705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:44449 every 8 connection(s)
I20260812 14:42:51.059015 29310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:51.059844 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 29310
I20260812 14:42:51.059988 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 28301
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.253@KRBTEST.COM
I20260812 14:42:51.080921 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.059827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46807 (local address 127.15.227.254:36191)
0812 14:42:51.060008 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.060011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.068840 (+ 8829us) server_negotiation.cc:407] Connection header received
0812 14:42:51.068916 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.068918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.068928 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.068952 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.071110 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.071709 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.072474 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.072669 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.072701 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.076136 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.076137 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.076138 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.076145 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.079455 (+ 3310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.080024 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.080025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.080026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.080084 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.080313 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.080314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.080314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.080423 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.080478 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.080746 (+ 268us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.080827 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260812 14:42:51.081254 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.059702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.193:46807)
0812 14:42:51.068795 (+ 9093us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.068803 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.068862 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.069005 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.069005 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.069159 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.069204 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.069206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.071788 (+ 2582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.071788 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.072405 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.072406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.072782 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.072783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.072901 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.074086 (+ 1185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.074097 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.076024 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.079637 (+ 3613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.079648 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.079651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.079938 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.080145 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.080145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.080146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.080238 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.080484 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.080491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.080620 (+ 129us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.080698 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.080846 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9068,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260812 14:42:51.081692 29710 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36191
I20260812 14:42:51.081789 29710 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.081983 29710 heartbeater.cc:511] Master 127.15.227.254:36191 requested a full tablet report, sending...
I20260812 14:42:51.082392 27934 ts_manager.cc:194] Re-registered known tserver with Master: 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449)
I20260812 14:42:51.082542 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.059764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:52533 (local address 127.15.227.253:42945)
0812 14:42:51.059882 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.059884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.067049 (+ 7165us) server_negotiation.cc:407] Connection header received
0812 14:42:51.067618 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.067619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.067631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.067657 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.069278 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.069884 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.073114 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.073343 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.073375 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.079520 (+ 6145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.079522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.079522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.079530 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.081178 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.081573 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.081575 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.081575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.081647 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.081898 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.081900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.081900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.082098 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.082166 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.082381 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.082463 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260812 14:42:51.082799 29725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.059702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.15.227.193:52533)
0812 14:42:51.066924 (+ 7222us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.066969 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.067515 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.067814 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.067817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.068447 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.068692 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.068698 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.069994 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.069995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.071010 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.071012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.073435 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.073436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.073487 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.078127 (+ 4640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.078133 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.079417 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.081269 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.081270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.081270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.081464 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.081730 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.081731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.081732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.081799 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.082163 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.082164 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.082217 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.082333 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.082440 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7177,"thread_start_us":87,"threads_started":1}
I20260812 14:42:51.082825 27934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:46807
I20260812 14:42:51.083511 29711 heartbeater.cc:347] Connected to a master server at 127.15.227.253:42945
I20260812 14:42:51.083556 29711 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.083627 29711 heartbeater.cc:511] Master 127.15.227.253:42945 requested a full tablet report, sending...
I20260812 14:42:51.083967 29049 ts_manager.cc:194] Registered new tserver with Master: 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449)
I20260812 14:42:51.085289 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:42803
--local_ip_for_outbound_sockets=127.15.227.194
--tserver_master_addrs=127.15.227.254:36191
--webserver_port=36247
--webserver_interface=127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab
--principal=oryx/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.252@KRBTEST.COM
I20260812 14:42:51.088929 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.060118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:40755 (local address 127.15.227.252:42215)
0812 14:42:51.073996 (+ 13878us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.073998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.074206 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:42:51.074235 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.074236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.074249 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.074307 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.074695 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.075593 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.076798 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.077175 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.077259 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.086483 (+ 9224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.086484 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.086485 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.086492 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.087973 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.088269 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.088270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.088270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.088335 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.088481 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.088482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.088482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.088607 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.088667 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.088759 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.088843 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13841,"thread_start_us":59,"threads_started":1}
I20260812 14:42:51.089041 29726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.059997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.15.227.193:40755)
0812 14:42:51.074074 (+ 14077us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.074085 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.074144 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.074354 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.074354 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.074564 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.074617 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.074618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.075718 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.075719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.076695 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.076698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.077334 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.077335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.077413 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.085094 (+ 7681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.085098 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.086400 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.088041 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.088042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.088042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.088207 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.088396 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.088396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.088397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.088439 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.088664 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.088665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.088694 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.088735 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.088809 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14059,"thread_start_us":63,"threads_started":1}
I20260812 14:42:51.089450 29708 heartbeater.cc:347] Connected to a master server at 127.15.227.252:42215
I20260812 14:42:51.089506 29708 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.089552 29708 heartbeater.cc:511] Master 127.15.227.252:42215 requested a full tablet report, sending...
I20260812 14:42:51.089941 29376 ts_manager.cc:194] Registered new tserver with Master: 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449)
W20260812 14:42:51.167708 29728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:51.167908 29728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:51.167929 29728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:51.169003 29728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:51.169036 29728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:51.169060 29728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:51.170485 29728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab
--principal=oryx/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:42803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=36247
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:51.170706 29728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:51.170931 29728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:51.175771 29736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:51.175746 29743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:51.175766 29738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:51.175961 29728 server_base.cc:1034] running on GCE node
I20260812 14:42:51.176157 29728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:51.176374 29728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:51.177531 29728 hybrid_clock.cc:648] HybridClock initialized: now 1786545771177519 us; error 27 us; skew 500 ppm
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:51.179962 29728 init.cc:377] Logged in from keytab as oryx/127.15.227.194@KRBTEST.COM (short username oryx)
I20260812 14:42:51.180361 29728 webserver.cc:460] Webserver started at http://127.15.227.194:36247/ using document root <none> and password file <none>
I20260812 14:42:51.180579 29728 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:51.180640 29728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:51.181857 29728 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:51.182443 29752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.182585 29728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.182667 29728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "3b46588679eb406796517b8da42aaa34"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "f3538e78832ec8926bb19457e100f4b4"
server_key_iv: "90ea09c7db3897f5ae8aeaebc5ae908e"
server_key_version: "encryptionkey@0"
I20260812 14:42:51.182951 29728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:51.196544 29728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:51.198123 29728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:51.198256 29728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:51.198436 29728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:51.198701 29728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:51.198734 29728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.198760 29728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:51.198782 29728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.203984 29728 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:42803
I20260812 14:42:51.204046 29982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:42803 every 8 connection(s)
I20260812 14:42:51.204335 29728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:51.211174 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 29728
I20260812 14:42:51.211330 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 28553
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.253@KRBTEST.COM
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.252@KRBTEST.COM
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:51.229628 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.205346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:50171 (local address 127.15.227.254:36191)
0812 14:42:51.205401 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.205403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.205595 (+ 192us) server_negotiation.cc:407] Connection header received
0812 14:42:51.206491 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.206493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.206507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.206559 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.207984 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.208458 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209253 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.209533 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209598 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.226497 (+ 16899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.226499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.226500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.226508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.228307 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.228760 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.228761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.228762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.228812 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.229017 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.229018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.229018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.229130 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.229162 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.229478 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.229548 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:42:51.230286 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.205230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:50171)
0812 14:42:51.205539 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.205546 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.206187 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.206586 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.206587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.206860 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.207024 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.207030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.208535 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.208536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.209168 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.209170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.209616 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.209754 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.210540 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.210544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.226364 (+ 15820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.228423 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.228431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.228433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.228690 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.228865 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.228865 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.228866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.228968 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.229194 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.229198 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.229349 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.229894 (+ 545us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.230085 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":4,"thread_start_us":62,"threads_started":1}
I20260812 14:42:51.232690 29984 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36191
I20260812 14:42:51.232800 29984 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.232986 29984 heartbeater.cc:511] Master 127.15.227.254:36191 requested a full tablet report, sending...
I20260812 14:42:51.233376 27934 ts_manager.cc:194] Re-registered known tserver with Master: 3b46588679eb406796517b8da42aaa34 (127.15.227.194:42803)
I20260812 14:42:51.234722 27934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:50171
I20260812 14:42:51.236075 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.204985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.15.227.194:56773)
0812 14:42:51.205423 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.205432 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.206192 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.206445 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.206446 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.206854 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.207024 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.207030 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.207973 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.207974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.209184 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.209187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.209619 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.209753 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.210466 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.210470 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.233025 (+ 22555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.235063 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.235065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.235273 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.235477 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.235478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.235529 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.235784 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.235785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.235824 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.235885 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.235980 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":65,"thread_start_us":93,"threads_started":1}
I20260812 14:42:51.236117 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.205015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:56773 (local address 127.15.227.252:42215)
0812 14:42:51.205066 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.205068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.205495 (+ 427us) server_negotiation.cc:407] Connection header received
0812 14:42:51.206360 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.206361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.206376 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.206427 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.207152 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.207881 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209301 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.209512 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209544 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.233159 (+ 23615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.233161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.233162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.233171 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.234964 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235357 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.235358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.235359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.235416 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235585 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.235586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.235586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.235705 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.235742 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.235932 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.236028 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:42:51.236634 29983 heartbeater.cc:347] Connected to a master server at 127.15.227.252:42215
I20260812 14:42:51.236690 29983 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.236744 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.205210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:52045 (local address 127.15.227.253:42945)
0812 14:42:51.205257 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.205258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.205267 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:51.206294 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.206296 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.206308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.206351 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.207110 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.207902 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209314 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.209600 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209670 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.233518 (+ 23848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.233520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.233520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.233529 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.235154 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235550 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.235552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.235552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.235611 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235793 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.235794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.235795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.235915 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.235973 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.236574 (+ 601us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.236664 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:42:51.236768 29983 heartbeater.cc:511] Master 127.15.227.252:42215 requested a full tablet report, sending...
I20260812 14:42:51.237025 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.204984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.15.227.194:52045)
0812 14:42:51.205158 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.205199 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.206179 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.206429 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.206432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.206836 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.207023 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.207028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.207973 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.207974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.209202 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.209204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.209720 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.209721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.209773 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.210282 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.210292 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.233379 (+ 23087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.235264 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.235265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.235464 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.235679 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.235680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.235680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.235727 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.235999 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.236000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.236041 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.236827 (+ 786us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.236940 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":43,"thread_start_us":95,"threads_started":1}
I20260812 14:42:51.237058 29376 ts_manager.cc:194] Registered new tserver with Master: 3b46588679eb406796517b8da42aaa34 (127.15.227.194:42803)
I20260812 14:42:51.237404 29987 heartbeater.cc:347] Connected to a master server at 127.15.227.253:42945
I20260812 14:42:51.237449 29987 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.237494 29987 heartbeater.cc:511] Master 127.15.227.253:42945 requested a full tablet report, sending...
I20260812 14:42:51.237761 29049 ts_manager.cc:194] Registered new tserver with Master: 3b46588679eb406796517b8da42aaa34 (127.15.227.194:42803)
I20260812 14:42:51.237932 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:45467
--local_ip_for_outbound_sockets=127.15.227.195
--tserver_master_addrs=127.15.227.254:36191
--webserver_port=34449
--webserver_interface=127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab
--principal=oryx/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:51.319434 29998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:51.319617 29998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:51.319645 29998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:51.320758 29998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:51.320796 29998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:51.320824 29998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:51.322355 29998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab
--principal=oryx/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:45467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=34449
--tserver_master_addrs=127.15.227.254:36191,127.15.227.253:42945,127.15.227.252:42215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:51.322561 29998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:51.322768 29998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:51.327536 30016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:51.327527 30012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:51.327543 30011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:51.327724 29998 server_base.cc:1034] running on GCE node
I20260812 14:42:51.327900 29998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:51.328081 29998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:51.329209 29998 hybrid_clock.cc:648] HybridClock initialized: now 1786545771329197 us; error 29 us; skew 500 ppm
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:51.331458 29998 init.cc:377] Logged in from keytab as oryx/127.15.227.195@KRBTEST.COM (short username oryx)
I20260812 14:42:51.331876 29998 webserver.cc:460] Webserver started at http://127.15.227.195:34449/ using document root <none> and password file <none>
I20260812 14:42:51.332078 29998 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:51.332147 29998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:51.333467 29998 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:51.334053 30029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.334205 29998 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:51.334280 29998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "4f770c70c2ff459abcd13c5e844a07be"
format_stamp: "Formatted at 2026-08-12 14:42:50 on dist-test-slave-q9h9"
server_key: "6caafd40a300a6b517451b73013048b6"
server_key_iv: "8de3f3bd6d5324f2cc511734c871af87"
server_key_version: "encryptionkey@0"
I20260812 14:42:51.334617 29998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:51.345465 29998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:51.347206 29998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:51.347322 29998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:51.347527 29998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:51.347797 29998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:51.347828 29998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.347854 29998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:51.347877 29998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:51.353377 29998 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:45467
I20260812 14:42:51.353413 30245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:45467 every 8 connection(s)
I20260812 14:42:51.353714 29998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.253@KRBTEST.COM
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
Aug 12 14:42:51 dist-test-slave-q9h9 krb5kdc[27840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545771, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.252@KRBTEST.COM
I20260812 14:42:51.362784 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 29998
I20260812 14:42:51.364629 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.354353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:48223 (local address 127.15.227.253:42945)
0812 14:42:51.354412 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.354414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.354643 (+ 229us) server_negotiation.cc:407] Connection header received
0812 14:42:51.355775 (+ 1132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.355777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.355789 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.355833 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.356505 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.357274 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.359249 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.359441 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.359498 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.361415 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.361416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.361417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.361424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.363104 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.363543 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.363545 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.363545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.363605 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.363896 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.363897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.363898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.364032 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.364076 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.364461 (+ 385us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.364535 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:42:51.365115 30254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.354351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.15.227.195:48223)
0812 14:42:51.354521 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.354560 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.355546 (+ 986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.355878 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.355879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.356222 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.356419 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.356424 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.357406 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.357406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.358741 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.358744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.359509 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.359510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.359587 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.359942 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.359946 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.361330 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.363190 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.363196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.363198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.363479 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.363677 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.363678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.363678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.363806 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.364142 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.364149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.364302 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.364737 (+ 435us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.364899 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":58,"thread_start_us":89,"threads_started":1}
I20260812 14:42:51.365144 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.354712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:38921 (local address 127.15.227.252:42215)
0812 14:42:51.354755 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.354757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.354859 (+ 102us) server_negotiation.cc:407] Connection header received
0812 14:42:51.355656 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.355657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.355670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.355699 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.357374 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.357961 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.358814 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.359103 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.359187 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.362378 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.362379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.362380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.362386 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.363829 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.364187 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.364189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.364189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.364250 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.364459 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.364461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.364461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.364586 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.364639 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.364975 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.365055 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:51.365427 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.354635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.15.227.195:38921)
0812 14:42:51.354798 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.354806 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.355557 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.355819 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.355823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.356229 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.356420 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.356426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.358052 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.358053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.358743 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.358745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.359205 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.359206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.359361 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.359841 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.359851 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.362274 (+ 2423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.363910 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.363911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.363912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.364104 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.364325 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.364326 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.364326 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.364387 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.364688 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.364689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.364849 (+ 160us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.365226 (+ 377us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.365331 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":10,"thread_start_us":65,"threads_started":1}
I20260812 14:42:51.365707 30251 heartbeater.cc:347] Connected to a master server at 127.15.227.253:42945
I20260812 14:42:51.365803 30251 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.365875 28043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.354338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:51741 (local address 127.15.227.254:36191)
0812 14:42:51.354462 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.354464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.354642 (+ 178us) server_negotiation.cc:407] Connection header received
0812 14:42:51.355655 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.355657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.355670 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.355722 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.356535 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.357278 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.358856 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.359139 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.359200 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.362318 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.362320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.362320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.362328 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.364796 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.365152 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.365153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.365153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.365210 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.365378 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.365379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.365380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.365491 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.365549 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.365724 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.365791 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 14:42:51.365996 30251 heartbeater.cc:511] Master 127.15.227.253:42945 requested a full tablet report, sending...
I20260812 14:42:51.366010 30247 heartbeater.cc:347] Connected to a master server at 127.15.227.252:42215
I20260812 14:42:51.366080 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.354351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.195:51741)
0812 14:42:51.354524 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.354560 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.355547 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.355819 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.355823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.356236 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:51.356420 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.356426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.357392 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.357393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.358750 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.358752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.359205 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.359206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.359361 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.360071 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:51.360076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:51.362248 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:51.364877 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.364878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.364878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.365056 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.365278 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:51.365279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:51.365279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:51.365322 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:51.365577 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:51.365577 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:51.365617 (+ 40us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.365685 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.365777 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":84,"thread_start_us":87,"threads_started":1}
I20260812 14:42:51.366073 30247 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.366199 30247 heartbeater.cc:511] Master 127.15.227.252:42215 requested a full tablet report, sending...
I20260812 14:42:51.366478 30248 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36191
I20260812 14:42:51.366492 29376 ts_manager.cc:194] Registered new tserver with Master: 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195:45467)
I20260812 14:42:51.366524 30248 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:51.366577 30248 heartbeater.cc:511] Master 127.15.227.254:36191 requested a full tablet report, sending...
I20260812 14:42:51.366678 29049 ts_manager.cc:194] Registered new tserver with Master: 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195:45467)
I20260812 14:42:51.366776 27934 ts_manager.cc:194] Re-registered known tserver with Master: 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195:45467)
I20260812 14:42:51.367130 27934 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:51741
I20260812 14:42:51.373263 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.364535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.15.227.252:42215)
0812 14:42:51.365209 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.365210 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.367225 (+ 2015us) server_negotiation.cc:407] Connection header received
0812 14:42:51.367337 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.367338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.367350 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.367401 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.367997 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.368494 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.369339 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.369589 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.369642 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.370761 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.370763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.370763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.370770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.372139 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.372471 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.372473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.372473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.372526 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.372737 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.372738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.372738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.372870 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.372971 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.373111 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.373178 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":49,"threads_started":1}
I20260812 14:42:51.373695 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.364754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.15.227.253:42945)
0812 14:42:51.364797 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.364799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.367402 (+ 2603us) server_negotiation.cc:407] Connection header received
0812 14:42:51.367478 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.367479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.367488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.367512 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.368111 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.368804 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.369563 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.369840 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.369900 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.371177 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.371178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.371178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.371185 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.372508 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.372841 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.372842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.372843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.372894 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.373060 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.373061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.373061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.373175 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.373256 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.373456 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.373616 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:42:51.374123 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.364412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.15.227.254:36191)
0812 14:42:51.365072 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.365074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.365080 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:42:51.365097 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.365098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.365109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.366439 (+ 1330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.367149 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.367861 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.368867 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.369180 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.369244 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.370626 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.370628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.370628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.370634 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.372377 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.373001 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.373002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.373002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.373059 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.373268 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.373269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.373269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.373589 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.373671 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.373942 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.374044 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":80,"threads_started":1}
I20260812 14:42:51.375401 27934 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:51.376856 27934 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:51.384007 29319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.380192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44449 (local address 127.0.0.1:46690)
0812 14:42:51.380433 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.380443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.380492 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.381009 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.381010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.381013 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:51.381065 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.381067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.382274 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.382275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.383044 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.383046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.383779 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.383780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.383840 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.383844 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.383905 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.383942 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222}
I20260812 14:42:51.384106 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.380280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:42803 (local address 127.0.0.1:46084)
0812 14:42:51.380660 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.380669 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.380703 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.381139 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.381140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.381142 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:51.381168 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.381169 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.381960 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.381961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.383060 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.383063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.383927 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.383928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.383980 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.383983 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.384019 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.384053 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":108,"threads_started":1}
I20260812 14:42:51.384287 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.380671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46690 (local address 127.15.227.193:44449)
0812 14:42:51.380866 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.380870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.380880 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:51.380916 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.380918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.380933 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.381009 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:51.381120 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.382177 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.383120 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.383701 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.383782 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.383838 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.384184 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.384227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
I20260812 14:42:51.384454 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.380149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:45467 (local address 127.0.0.1:56108)
0812 14:42:51.380433 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.380443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.380496 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.380898 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.380899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.380901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:51.380942 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.380943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.382252 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.382253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.383336 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.383339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.384220 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.384221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.384297 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.384301 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.384355 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.384391 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":27}
I20260812 14:42:51.384636 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.380220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.15.227.195:45467)
0812 14:42:51.380382 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.380386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.380488 (+ 102us) server_negotiation.cc:407] Connection header received
0812 14:42:51.380557 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.380559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.380593 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.381011 (+ 418us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:51.381058 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.382153 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.383418 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.384067 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.384142 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.384188 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.384525 (+ 337us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.384564 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260812 14:42:51.384651 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.380815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.15.227.194:42803)
0812 14:42:51.380972 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.380976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.380986 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:51.381020 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.381022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.381037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.381222 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:51.381272 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.381888 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.383156 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.383859 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.383945 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.383997 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.384030 (+ 33us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.384445 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260812 14:42:51.384948 29579 tablet_service.cc:1467] Processing CreateTablet for tablet 5449adf23e6f40e3894cbdb9d3db79ac (DEFAULT_TABLE table=test-table [id=26b5002635d34874a9943d0affa1066f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:51.385161 30120 tablet_service.cc:1467] Processing CreateTablet for tablet 5449adf23e6f40e3894cbdb9d3db79ac (DEFAULT_TABLE table=test-table [id=26b5002635d34874a9943d0affa1066f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:51.385172 29856 tablet_service.cc:1467] Processing CreateTablet for tablet 5449adf23e6f40e3894cbdb9d3db79ac (DEFAULT_TABLE table=test-table [id=26b5002635d34874a9943d0affa1066f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:51.385329 29579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5449adf23e6f40e3894cbdb9d3db79ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:51.385488 30120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5449adf23e6f40e3894cbdb9d3db79ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:51.385500 29856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5449adf23e6f40e3894cbdb9d3db79ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:51.388080 30293 tablet_bootstrap.cc:492] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Bootstrap starting.
I20260812 14:42:51.388677 30300 tablet_bootstrap.cc:492] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Bootstrap starting.
I20260812 14:42:51.388711 30293 tablet_bootstrap.cc:654] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:51.389156 30293 log.cc:824] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:51.389298 30300 tablet_bootstrap.cc:654] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:51.389556 30300 log.cc:824] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:51.389868 30294 tablet_bootstrap.cc:492] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Bootstrap starting.
I20260812 14:42:51.389937 30293 tablet_bootstrap.cc:492] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: No bootstrap required, opened a new log
I20260812 14:42:51.390156 30293 ts_tablet_manager.cc:1397] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:42:51.390285 30300 tablet_bootstrap.cc:492] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: No bootstrap required, opened a new log
I20260812 14:42:51.390353 30300 ts_tablet_manager.cc:1397] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:51.390479 30294 tablet_bootstrap.cc:654] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:51.390822 30294 log.cc:824] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:51.391505 30294 tablet_bootstrap.cc:492] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: No bootstrap required, opened a new log
I20260812 14:42:51.391569 30294 ts_tablet_manager.cc:1397] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:51.392490 30293 raft_consensus.cc:348] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.392635 30293 raft_consensus.cc:374] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:51.392668 30293 raft_consensus.cc:729] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b46588679eb406796517b8da42aaa34, State: Initialized, Role: FOLLOWER
I20260812 14:42:51.392800 30293 consensus_queue.cc:260] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.392850 30300 raft_consensus.cc:348] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.392999 30300 raft_consensus.cc:374] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:51.393036 30300 raft_consensus.cc:729] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f770c70c2ff459abcd13c5e844a07be, State: Initialized, Role: FOLLOWER
I20260812 14:42:51.393049 30293 ts_tablet_manager.cc:1428] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:42:51.393137 30300 consensus_queue.cc:260] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.393373 30300 ts_tablet_manager.cc:1428] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:42:51.393476 30248 heartbeater.cc:503] Master 127.15.227.254:36191 was elected leader, sending a full tablet report...
I20260812 14:42:51.393886 29984 heartbeater.cc:503] Master 127.15.227.254:36191 was elected leader, sending a full tablet report...
I20260812 14:42:51.394074 30294 raft_consensus.cc:348] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.394218 30294 raft_consensus.cc:374] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:51.394251 30294 raft_consensus.cc:729] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707d1c53fc2d42fd873b6f288c5ae04d, State: Initialized, Role: FOLLOWER
I20260812 14:42:51.394373 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.390245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.0.0.1:40402)
0812 14:42:51.390371 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.390381 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.390427 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.390591 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.390592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.390593 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:51.390625 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.390627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.391553 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.391554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.392589 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.392592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.394163 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.394164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.394222 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.394226 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.394276 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.394313 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260812 14:42:51.394641 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.390304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40402 (local address 127.15.227.252:42215)
0812 14:42:51.390343 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.390345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.390478 (+ 133us) server_negotiation.cc:407] Connection header received
0812 14:42:51.390501 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.390502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.390524 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.390581 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:51.390717 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.391411 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.393209 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.393757 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.393819 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.393877 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.394537 (+ 660us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.394576 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:42:51.394832 30294 consensus_queue.cc:260] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.394899 29395 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260812 14:42:51.395092 30294 ts_tablet_manager.cc:1428] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:42:51.395298 29710 heartbeater.cc:503] Master 127.15.227.254:36191 was elected leader, sending a full tablet report...
I20260812 14:42:51.395743 29440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 205e92ff322144cb919d11bfae3bdf89. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:51.395828 29440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: This master's current role is: LEARNER
I20260812 14:42:51.438429 30307 raft_consensus.cc:482] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:51.438537 30307 raft_consensus.cc:504] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.438832 30307 leader_election.cc:288] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449), 3b46588679eb406796517b8da42aaa34 (127.15.227.194:42803)
I20260812 14:42:51.441499 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.439076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:57963 (local address 127.15.227.193:44449)
0812 14:42:51.439127 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.439129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.439138 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:51.439165 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.439166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.439175 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.439226 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:51.439412 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.439913 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.440842 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.441208 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.441272 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.441281 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.441400 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.441435 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:42:51.441522 30256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.438998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44449 (local address 127.15.227.195:57963)
0812 14:42:51.439039 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.439051 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.439100 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.439280 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.439281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.439283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:51.439347 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.439348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.439987 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.439989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.440758 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.440761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.441281 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.441282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.441344 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.441346 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.441390 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.441429 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 14:42:51.441905 29619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5449adf23e6f40e3894cbdb9d3db79ac" candidate_uuid: "4f770c70c2ff459abcd13c5e844a07be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" is_pre_election: true
I20260812 14:42:51.441967 30259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.439150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:42803 (local address 127.15.227.195:49395)
0812 14:42:51.439183 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:51.439191 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:51.439221 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:51.439353 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:51.439354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:51.439355 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:51.439413 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.439415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.440439 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.440441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.441160 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:51.441162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.441767 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.441769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:51.441844 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.441847 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:51.441893 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:42:51.441924 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260812 14:42:51.442034 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.439217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:49395 (local address 127.15.227.194:42803)
0812 14:42:51.439260 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.439262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.439268 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:42:51.439292 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.439292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.439301 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.439344 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:51.439488 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.440364 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.441244 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.441633 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.441695 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.441703 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.441912 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.441953 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:42:51.442094 29619 raft_consensus.cc:2454] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f770c70c2ff459abcd13c5e844a07be in term 0.
I20260812 14:42:51.442310 30032 leader_election.cc:302] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f770c70c2ff459abcd13c5e844a07be, 707d1c53fc2d42fd873b6f288c5ae04d; no voters:
I20260812 14:42:51.442296 29896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5449adf23e6f40e3894cbdb9d3db79ac" candidate_uuid: "4f770c70c2ff459abcd13c5e844a07be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b46588679eb406796517b8da42aaa34" is_pre_election: true
I20260812 14:42:51.442435 29896 raft_consensus.cc:2454] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f770c70c2ff459abcd13c5e844a07be in term 0.
I20260812 14:42:51.442442 30307 raft_consensus.cc:2781] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:51.442478 30307 raft_consensus.cc:482] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:51.442497 30307 raft_consensus.cc:3037] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:51.443156 30307 raft_consensus.cc:504] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.443264 30307 leader_election.cc:288] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [CANDIDATE]: Term 1 election: Requested vote from peers 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449), 3b46588679eb406796517b8da42aaa34 (127.15.227.194:42803)
I20260812 14:42:51.443439 29619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5449adf23e6f40e3894cbdb9d3db79ac" candidate_uuid: "4f770c70c2ff459abcd13c5e844a07be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707d1c53fc2d42fd873b6f288c5ae04d"
I20260812 14:42:51.443467 29896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5449adf23e6f40e3894cbdb9d3db79ac" candidate_uuid: "4f770c70c2ff459abcd13c5e844a07be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b46588679eb406796517b8da42aaa34"
I20260812 14:42:51.443502 29619 raft_consensus.cc:3037] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:51.443531 29896 raft_consensus.cc:3037] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:51.444202 29619 raft_consensus.cc:2454] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f770c70c2ff459abcd13c5e844a07be in term 1.
I20260812 14:42:51.444245 29896 raft_consensus.cc:2454] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f770c70c2ff459abcd13c5e844a07be in term 1.
I20260812 14:42:51.444370 30032 leader_election.cc:302] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b46588679eb406796517b8da42aaa34, 4f770c70c2ff459abcd13c5e844a07be; no voters:
I20260812 14:42:51.444453 30307 raft_consensus.cc:2781] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:51.444582 30307 raft_consensus.cc:686] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 1 LEADER]: Becoming Leader. State: Replica: 4f770c70c2ff459abcd13c5e844a07be, State: Running, Role: LEADER
I20260812 14:42:51.444667 30307 consensus_queue.cc:237] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } }
I20260812 14:42:51.445461 27930 catalog_manager.cc:5495] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195). New cstate: current_term: 1 leader_uuid: "4f770c70c2ff459abcd13c5e844a07be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:51.446139 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: NON_VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: true } } }
I20260812 14:42:51.446228 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:51.446933 30323 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: attempting to promote NON_VOTER 6ebdf2c2baf74bfcba8af5bf8abef14f to VOTER
I20260812 14:42:51.447086 30323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } }
I20260812 14:42:51.447361 28987 raft_consensus.cc:1013] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 LEADER]: attempt to promote peer 6ebdf2c2baf74bfcba8af5bf8abef14f: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260812 14:42:51.447598 29395 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 LEARNER]: Refusing update from remote peer 205e92ff322144cb919d11bfae3bdf89: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 14:42:51.447677 29071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Refusing update from remote peer 205e92ff322144cb919d11bfae3bdf89: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260812 14:42:51.447865 30330 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 14:42:51.447976 30323 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260812 14:42:51.448650 30323 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.448803 29071 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.448807 29395 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.449609 29440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.449623 30323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.449692 29440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:42:51.449698 30323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:51.449775 30323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.449831 30323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:51.449894 30323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.449816 30335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "205e92ff322144cb919d11bfae3bdf89" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:51.449949 30323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:51.449966 30335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:42:51.455554 29989 tablet.cc:2369] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:51.461153 30286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.458962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.15.227.195:45467)
0812 14:42:51.459006 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.459008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.459017 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:51.459069 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.459070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.459088 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.459153 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:51.459282 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.459740 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.460458 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.460658 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.460720 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.460865 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:51.460963 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:51.461032 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.461067 (+ 35us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.461085 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:42:51.462747 29896 raft_consensus.cc:1261] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 1 FOLLOWER]: Refusing update from remote peer 4f770c70c2ff459abcd13c5e844a07be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:51.462771 29619 raft_consensus.cc:1261] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 1 FOLLOWER]: Refusing update from remote peer 4f770c70c2ff459abcd13c5e844a07be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:51.463073 30318 consensus_queue.cc:1035] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:51.463186 30307 consensus_queue.cc:1035] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [LEADER]: Connected to new peer: Peer: permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:51.465794 30338 mvcc.cc:205] Tried to move back new op lower bound from 7317691479909216256 to 7317691479838003200. Current Snapshot: MvccSnapshot[applied={T|T < 7317691479909216256}]
I20260812 14:42:51.466168 30341 mvcc.cc:205] Tried to move back new op lower bound from 7317691479909216256 to 7317691479838003200. Current Snapshot: MvccSnapshot[applied={T|T < 7317691479909216256}]
I20260812 14:42:51.467564 30339 mvcc.cc:205] Tried to move back new op lower bound from 7317691479909216256 to 7317691479838003200. Current Snapshot: MvccSnapshot[applied={T|T < 7317691479909216256}]
I20260812 14:42:51.468686 27930 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:51.468761 27930 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:51.470286 27930 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 205e92ff322144cb919d11bfae3bdf89: Sending DeleteTablet for 3 replicas of tablet 5449adf23e6f40e3894cbdb9d3db79ac
I20260812 14:42:51.470607 29856 tablet_service.cc:1514] Processing DeleteTablet for tablet 5449adf23e6f40e3894cbdb9d3db79ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:51 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:46084
I20260812 14:42:51.470613 30120 tablet_service.cc:1514] Processing DeleteTablet for tablet 5449adf23e6f40e3894cbdb9d3db79ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:51 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:56108
I20260812 14:42:51.470880 29579 tablet_service.cc:1514] Processing DeleteTablet for tablet 5449adf23e6f40e3894cbdb9d3db79ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:51 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:46690
I20260812 14:42:51.470907 30352 tablet_replica.cc:330] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: stopping tablet replica
I20260812 14:42:51.471002 30352 raft_consensus.cc:2229] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:51.471086 30352 raft_consensus.cc:2258] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:51.471524 30352 ts_tablet_manager.cc:1905] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:51.471738 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 27866
I20260812 14:42:51.481081 30352 ts_tablet_manager.cc:1918] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:51.481156 30352 log.cc:1196] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/5449adf23e6f40e3894cbdb9d3db79ac
I20260812 14:42:51.481392 30352 ts_tablet_manager.cc:1939] T 5449adf23e6f40e3894cbdb9d3db79ac P 4f770c70c2ff459abcd13c5e844a07be: Deleting consensus metadata
I20260812 14:42:51.482383 30353 tablet_replica.cc:330] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: stopping tablet replica
I20260812 14:42:51.482571 30356 tablet_replica.cc:330] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: stopping tablet replica
I20260812 14:42:51.482736 30356 raft_consensus.cc:2229] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:51.482892 30353 raft_consensus.cc:2229] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:51.483058 30356 raft_consensus.cc:2258] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:51.483227 30353 raft_consensus.cc:2258] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:51.483877 30353 ts_tablet_manager.cc:1905] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:51.484311 30356 ts_tablet_manager.cc:1905] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:51.485973 30353 ts_tablet_manager.cc:1918] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:51.486065 30353 log.cc:1196] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/5449adf23e6f40e3894cbdb9d3db79ac
I20260812 14:42:51.486303 30353 ts_tablet_manager.cc:1939] T 5449adf23e6f40e3894cbdb9d3db79ac P 3b46588679eb406796517b8da42aaa34: Deleting consensus metadata
W20260812 14:42:51.487345 29753 connection.cc:535] server connection from 127.0.0.1:46084 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 14:42:51.487571 30030 connection.cc:535] server connection from 127.0.0.1:56108 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:42:51.488277 30356 ts_tablet_manager.cc:1918] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:51.488353 30356 log.cc:1196] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/5449adf23e6f40e3894cbdb9d3db79ac
I20260812 14:42:51.488610 30356 ts_tablet_manager.cc:1939] T 5449adf23e6f40e3894cbdb9d3db79ac P 707d1c53fc2d42fd873b6f288c5ae04d: Deleting consensus metadata
W20260812 14:42:51.488941 29497 connection.cc:441] server connection from 127.0.0.1:46690 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46690 (request call id 1) could send its response
I20260812 14:42:51.492995 29727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.471414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.15.227.252:42215)
0812 14:42:51.471454 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.471456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.482241 (+ 10785us) server_negotiation.cc:407] Connection header received
0812 14:42:51.482298 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.482300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.482312 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.482360 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.483137 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.483838 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.484827 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.485107 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.485153 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.490364 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.490365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.490366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.490373 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.491893 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.492264 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.492265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.492265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.492322 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.492499 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.492500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.492500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.492618 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.492711 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.492821 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.492901 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:51.496387 29141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.489073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.15.227.253:42945)
0812 14:42:51.489116 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.489118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.489252 (+ 134us) server_negotiation.cc:407] Connection header received
0812 14:42:51.489355 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.489356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.489368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.489398 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.490163 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.490855 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.491850 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.492135 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.492222 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.493909 (+ 1687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.493910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.493910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.493917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.495235 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.495509 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.495510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.495510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.495564 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.495917 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.495918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.495919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.496021 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.496100 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.496223 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.496288 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:42:51.497470 30277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:51.489541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40424 (local address 127.15.227.252:42215)
0812 14:42:51.489582 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:51.489584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:51.489872 (+ 288us) server_negotiation.cc:407] Connection header received
0812 14:42:51.489961 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:51.489962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:51.489973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:51.490010 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:51.491390 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.492100 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.493248 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:51.493555 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:51.493595 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:51.494943 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:51.494945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:51.494945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:51.494952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:51.496490 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.496832 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.496833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.496833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.496889 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:51.497045 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:51.497045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:51.497045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:51.497161 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:51.497246 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:51.497305 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:42:51.497382 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:42:52.468116 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.467822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.195:47975)
0812 14:42:52.467963 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.468058 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
W20260812 14:42:52.468372 30248 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:36191 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
I20260812 14:42:52.468454 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.468211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.193:52119)
0812 14:42:52.468322 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.468410 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":51,"threads_started":1}
W20260812 14:42:52.468796 29710 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:36191 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
I20260812 14:42:52.469167 30377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.468955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:54777)
0812 14:42:52.469058 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.469134 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
W20260812 14:42:52.469344 29984 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:36191 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
I20260812 14:42:52.970790 30378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 205e92ff322144cb919d11bfae3bdf89)
I20260812 14:42:52.970929 30378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } }
I20260812 14:42:52.971160 30378 leader_election.cc:288] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 205e92ff322144cb919d11bfae3bdf89 (127.15.227.254:36191), 095f21efeb3e4a65a13ba79a4fe021c3 (127.15.227.253:42945)
I20260812 14:42:52.971189 30381 raft_consensus.cc:482] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 205e92ff322144cb919d11bfae3bdf89)
I20260812 14:42:52.971307 30381 raft_consensus.cc:504] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } }
I20260812 14:42:52.971525 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.971240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53568)
0812 14:42:52.971438 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.971487 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":39,"threads_started":1}
I20260812 14:42:52.971547 30381 leader_election.cc:288] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 205e92ff322144cb919d11bfae3bdf89 (127.15.227.254:36191), 6ebdf2c2baf74bfcba8af5bf8abef14f (127.15.227.252:42215)
W20260812 14:42:52.971681 29353 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
I20260812 14:42:52.972095 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.971994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53584)
0812 14:42:52.972036 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.972068 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 14:42:52.972306 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.971994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53580)
0812 14:42:52.972221 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.972265 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":57,"threads_started":1}
W20260812 14:42:52.972371 29353 leader_election.cc:334] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 205e92ff322144cb919d11bfae3bdf89 (127.15.227.254:36191): Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
W20260812 14:42:52.972407 29030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
I20260812 14:42:52.972775 30393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.972687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53590)
0812 14:42:52.972719 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.972749 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260812 14:42:52.972882 29030 leader_election.cc:334] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 205e92ff322144cb919d11bfae3bdf89 (127.15.227.254:36191): Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
I20260812 14:42:52.974422 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.971376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.15.227.253:42945)
0812 14:42:52.971497 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:52.971499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:52.971613 (+ 114us) server_negotiation.cc:407] Connection header received
0812 14:42:52.971642 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:52.971642 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:52.971652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:52.971713 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:52.971863 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.972592 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.973634 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.974109 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.974193 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:52.974203 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:52.974352 (+ 149us) server_negotiation.cc:299] Negotiation successful
0812 14:42:52.974380 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":47,"threads_started":1}
I20260812 14:42:52.974452 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.971324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:42945 (local address 127.0.0.1:34400)
0812 14:42:52.971482 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.971494 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:52.971554 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:52.971728 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:52.971729 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:52.971732 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:52.971790 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:52.971792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.972681 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.972682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:52.973550 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:52.973553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.974217 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.974218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:52.974289 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:52.974294 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:42:52.974348 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:42:52.974385 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":59,"threads_started":1}
I20260812 14:42:52.974676 29071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" is_pre_election: true
I20260812 14:42:52.974771 29071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ebdf2c2baf74bfcba8af5bf8abef14f in term 1.
I20260812 14:42:52.974936 29351 leader_election.cc:302] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 095f21efeb3e4a65a13ba79a4fe021c3, 6ebdf2c2baf74bfcba8af5bf8abef14f; no voters: 205e92ff322144cb919d11bfae3bdf89
I20260812 14:42:52.975031 30378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 14:42:52.975070 30378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 205e92ff322144cb919d11bfae3bdf89)
I20260812 14:42:52.975088 30378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:42:52.975373 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.972254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40440 (local address 127.15.227.252:42215)
0812 14:42:52.972368 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:52.972370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:52.972590 (+ 220us) server_negotiation.cc:407] Connection header received
0812 14:42:52.972686 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:52.972687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:52.972697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:52.972764 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:52.972992 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.973463 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.974547 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.975035 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.975080 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:52.975087 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:52.975270 (+ 183us) server_negotiation.cc:299] Negotiation successful
0812 14:42:52.975311 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":96,"thread_start_us":71,"threads_started":1}
I20260812 14:42:52.975356 30392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.972142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:42215 (local address 127.0.0.1:40440)
0812 14:42:52.972528 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.972538 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:52.972613 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:52.972770 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:52.972771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:52.972774 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:52.972815 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:52.972816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.973532 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.973533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:52.974456 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:52.974459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:52.975142 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:52.975143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:52.975206 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:52.975209 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:52.975249 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 14:42:52.975291 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":38,"threads_started":1}
I20260812 14:42:52.975577 29395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" is_pre_election: true
I20260812 14:42:52.975942 30378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } }
I20260812 14:42:52.976056 29395 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 095f21efeb3e4a65a13ba79a4fe021c3 in current term 2: Already voted for candidate 6ebdf2c2baf74bfcba8af5bf8abef14f in this term.
I20260812 14:42:52.976068 30378 leader_election.cc:288] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [CANDIDATE]: Term 2 election: Requested vote from peers 205e92ff322144cb919d11bfae3bdf89 (127.15.227.254:36191), 095f21efeb3e4a65a13ba79a4fe021c3 (127.15.227.253:42945)
I20260812 14:42:52.976230 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.976151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53596)
0812 14:42:52.976179 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.976211 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260812 14:42:52.976257 29030 leader_election.cc:302] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 095f21efeb3e4a65a13ba79a4fe021c3; no voters: 205e92ff322144cb919d11bfae3bdf89, 6ebdf2c2baf74bfcba8af5bf8abef14f
I20260812 14:42:52.976296 29071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "095f21efeb3e4a65a13ba79a4fe021c3"
I20260812 14:42:52.976351 30381 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:42:52.976486 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:52.976407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53610)
0812 14:42:52.976439 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:52.976464 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260812 14:42:52.976553 29353 leader_election.cc:334] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 205e92ff322144cb919d11bfae3bdf89 (127.15.227.254:36191): Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
I20260812 14:42:52.977012 30381 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:42:52.977582 29071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ebdf2c2baf74bfcba8af5bf8abef14f in term 2.
I20260812 14:42:52.977736 29351 leader_election.cc:302] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 095f21efeb3e4a65a13ba79a4fe021c3, 6ebdf2c2baf74bfcba8af5bf8abef14f; no voters: 205e92ff322144cb919d11bfae3bdf89
I20260812 14:42:52.977813 30378 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:42:52.977928 30378 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [term 2 LEADER]: Becoming Leader. State: Replica: 6ebdf2c2baf74bfcba8af5bf8abef14f, State: Running, Role: LEADER
I20260812 14:42:52.977989 30378 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } }
I20260812 14:42:52.978302 30399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ebdf2c2baf74bfcba8af5bf8abef14f. Latest consensus state: current_term: 2 leader_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:52.978374 30399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:52.978508 30404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:52.978929 30404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:52.979005 30404 catalog_manager.cc:1259] Loaded cluster ID: 7132973b1cd54dfa99e0f4a82d790bf2
I20260812 14:42:52.979036 30404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:52.979180 30404 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:52.979246 30404 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Loaded TSK: 0
I20260812 14:42:52.979377 30404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:42:53.025641 29374 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:53.027426 29374 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:53.028573 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.028473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53628)
0812 14:42:53.028518 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.028545 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 14:42:53.028591 29071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [term 2 FOLLOWER]: Refusing update from remote peer 6ebdf2c2baf74bfcba8af5bf8abef14f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 14:42:53.028936 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.028841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.0.0.1:53642)
0812 14:42:53.028889 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.028918 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 14:42:53.028895 30378 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [LEADER]: Connected to new peer: Peer: permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 14:42:53.029024 29353 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f -> Peer 205e92ff322144cb919d11bfae3bdf89 (127.15.227.254:36191): Couldn't send request to peer 205e92ff322144cb919d11bfae3bdf89. Status: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 14:42:53.029512 30381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ebdf2c2baf74bfcba8af5bf8abef14f. Latest consensus state: current_term: 2 leader_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:53.029593 30381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:42:53.030154 30399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:53.030241 30399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:53.030364 30381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "205e92ff322144cb919d11bfae3bdf89" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36191 } } peers { permanent_uuid: "095f21efeb3e4a65a13ba79a4fe021c3" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 42945 } attrs { promote: false } } peers { permanent_uuid: "6ebdf2c2baf74bfcba8af5bf8abef14f" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 42215 } attrs { promote: false } } }
I20260812 14:42:53.030455 30381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095f21efeb3e4a65a13ba79a4fe021c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:42:53.030534 30407 mvcc.cc:205] Tried to move back new op lower bound from 7317691486322831360 to 7317691486118535168. Current Snapshot: MvccSnapshot[applied={T|T < 7317691486322831360}]
I20260812 14:42:53.035223 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.032048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44449 (local address 127.0.0.1:46696)
0812 14:42:53.032141 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.032150 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:53.032196 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:53.032416 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:53.032418 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:53.032420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:53.032478 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.032480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.033211 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.033213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.034344 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.034347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.035000 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.035001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.035072 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.035076 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:53.035122 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:42:53.035161 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 14:42:53.035235 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.032148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.15.227.193:44449)
0812 14:42:53.032280 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:53.032282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:53.032290 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:53.032315 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:53.032316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:53.032328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:53.032540 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:53.032577 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.033142 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.034435 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.034934 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.035002 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.035011 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:53.035134 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 14:42:53.035172 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260812 14:42:53.035465 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.031963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:42803 (local address 127.0.0.1:46100)
0812 14:42:53.032003 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.032013 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:53.032051 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:53.032574 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:53.032575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:53.032577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:53.032621 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.032622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.033593 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.033594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.034495 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.034498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.035253 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.035254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.035312 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.035317 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:42:53.035346 (+ 29us) client_negotiation.cc:240] Negotiation successful
0812 14:42:53.035397 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":1}
I20260812 14:42:53.035579 29579 tablet_service.cc:1467] Processing CreateTablet for tablet 268ddca4079749edb6c282f4e1c91dec (DEFAULT_TABLE table=test-table [id=ebf890a9c49b4019a75af9c49c522e1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:53.035614 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.032309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56122 (local address 127.15.227.195:45467)
0812 14:42:53.032430 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:53.032431 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:53.032438 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:53.032462 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:53.032463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:53.032472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:53.032527 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:53.032667 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.033407 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.034690 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.035253 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.035313 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.035321 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:53.035506 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 14:42:53.035548 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260812 14:42:53.035727 29579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 268ddca4079749edb6c282f4e1c91dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:53.035737 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.032077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:45467 (local address 127.0.0.1:56122)
0812 14:42:53.032330 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.032340 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:53.032393 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:53.032570 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:53.032571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:53.032573 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:53.032603 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.032605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.033492 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.033493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.034622 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.034624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.035350 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.035351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.035398 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.035402 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:53.035443 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 14:42:53.035478 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":112,"threads_started":1}
I20260812 14:42:53.035734 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.032038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.15.227.194:42803)
0812 14:42:53.032172 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:53.032174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:53.032182 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:53.032494 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:53.032497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:53.032516 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:53.032573 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:53.032691 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.033533 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.034565 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.035158 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.035579 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.035588 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:53.035605 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 14:42:53.035659 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
I20260812 14:42:53.035986 29856 tablet_service.cc:1467] Processing CreateTablet for tablet 268ddca4079749edb6c282f4e1c91dec (DEFAULT_TABLE table=test-table [id=ebf890a9c49b4019a75af9c49c522e1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:53.036094 29856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 268ddca4079749edb6c282f4e1c91dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:53.037062 30419 tablet_bootstrap.cc:492] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d: Bootstrap starting.
I20260812 14:42:53.037196 30120 tablet_service.cc:1467] Processing CreateTablet for tablet 268ddca4079749edb6c282f4e1c91dec (DEFAULT_TABLE table=test-table [id=ebf890a9c49b4019a75af9c49c522e1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:53.037328 30120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 268ddca4079749edb6c282f4e1c91dec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:53.037338 30420 tablet_bootstrap.cc:492] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: Bootstrap starting.
I20260812 14:42:53.037492 30419 tablet_bootstrap.cc:654] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:53.037658 30420 tablet_bootstrap.cc:654] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:53.038357 30420 tablet_bootstrap.cc:492] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: No bootstrap required, opened a new log
I20260812 14:42:53.038401 30420 ts_tablet_manager.cc:1397] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:53.038522 30420 raft_consensus.cc:348] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.038580 30420 raft_consensus.cc:374] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:53.038594 30420 raft_consensus.cc:729] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b46588679eb406796517b8da42aaa34, State: Initialized, Role: FOLLOWER
I20260812 14:42:53.038651 30420 consensus_queue.cc:260] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.038669 30428 tablet_bootstrap.cc:492] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Bootstrap starting.
I20260812 14:42:53.038861 30420 ts_tablet_manager.cc:1428] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:53.039028 30428 tablet_bootstrap.cc:654] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:53.039867 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.039183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:50347)
0812 14:42:53.039768 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.039802 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":79,"threads_started":1}
I20260812 14:42:53.039924 30428 tablet_bootstrap.cc:492] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: No bootstrap required, opened a new log
I20260812 14:42:53.039968 30428 ts_tablet_manager.cc:1397] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:53.040033 29983 heartbeater.cc:503] Master 127.15.227.252:42215 was elected leader, sending a full tablet report...
I20260812 14:42:53.040074 30428 raft_consensus.cc:348] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.040144 30428 raft_consensus.cc:374] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:53.040162 30428 raft_consensus.cc:729] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f770c70c2ff459abcd13c5e844a07be, State: Initialized, Role: FOLLOWER
I20260812 14:42:53.040201 30428 consensus_queue.cc:260] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.040342 30428 ts_tablet_manager.cc:1428] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:53.040802 30247 heartbeater.cc:503] Master 127.15.227.252:42215 was elected leader, sending a full tablet report...
I20260812 14:42:53.040836 30419 tablet_bootstrap.cc:492] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d: No bootstrap required, opened a new log
I20260812 14:42:53.040881 30419 ts_tablet_manager.cc:1397] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:42:53.040952 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.040601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.195:47765)
0812 14:42:53.040795 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.040829 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":72,"threads_started":1}
I20260812 14:42:53.040997 30419 raft_consensus.cc:348] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.041065 30419 raft_consensus.cc:374] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:53.041086 30419 raft_consensus.cc:729] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707d1c53fc2d42fd873b6f288c5ae04d, State: Initialized, Role: FOLLOWER
I20260812 14:42:53.041122 30419 consensus_queue.cc:260] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.041245 30419 ts_tablet_manager.cc:1428] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:53.041302 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.041236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.195:41837)
0812 14:42:53.041253 (+ 17us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.041277 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6,"mutex_wait_us":2}
I20260812 14:42:53.040680 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.040261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:54157)
0812 14:42:53.040525 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.040554 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251}
I20260812 14:42:53.041772 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.041569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.193:58821)
0812 14:42:53.041696 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.041732 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260812 14:42:53.041985 29708 heartbeater.cc:503] Master 127.15.227.252:42215 was elected leader, sending a full tablet report...
I20260812 14:42:53.042235 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.042152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.193:56445)
0812 14:42:53.042182 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.042211 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260812 14:42:53.074853 29713 tablet.cc:2369] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:42:53.105232 30253 tablet.cc:2369] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:53.129089 30430 raft_consensus.cc:482] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:53.129161 30430 raft_consensus.cc:504] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.129493 30430 leader_election.cc:288] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449), 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195:45467)
I20260812 14:42:53.133277 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.129764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:60239 (local address 127.15.227.195:45467)
0812 14:42:53.129816 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:53.129818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:53.129833 (+ 15us) server_negotiation.cc:407] Connection header received
0812 14:42:53.129908 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:53.129909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:53.129919 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:53.129982 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:53.130173 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.130928 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.132348 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.132917 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.133000 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.133011 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:53.133162 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:42:53.133207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:42:53.133277 30417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.129780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:45237 (local address 127.15.227.193:44449)
0812 14:42:53.129843 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:53.129845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:53.130064 (+ 219us) server_negotiation.cc:407] Connection header received
0812 14:42:53.130105 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:53.130106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:53.130115 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:53.130176 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:53.130253 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.130941 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.132347 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.132917 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.133000 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.133011 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:53.133161 (+ 150us) server_negotiation.cc:299] Negotiation successful
0812 14:42:53.133215 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:42:53.133281 30434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.129690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:45467 (local address 127.15.227.194:60239)
0812 14:42:53.129777 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.129789 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:53.129847 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:53.130010 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:53.130029 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:53.130033 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:53.130105 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.130106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.131005 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.131006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.132252 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.132255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.133001 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.133002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.133092 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.133096 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:53.133147 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:42:53.133198 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":61}
I20260812 14:42:53.133291 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.129690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44449 (local address 127.15.227.194:45237)
0812 14:42:53.129980 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.129988 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:53.130061 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:53.130178 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:53.130178 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:53.130179 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:53.130208 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.130209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.131008 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.131008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.132252 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:53.132255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.132998 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.132999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:53.133092 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.133096 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:53.133147 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:42:53.133198 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":140,"thread_start_us":135,"threads_started":1}
I20260812 14:42:53.133533 30158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "268ddca4079749edb6c282f4e1c91dec" candidate_uuid: "3b46588679eb406796517b8da42aaa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f770c70c2ff459abcd13c5e844a07be" is_pre_election: true
I20260812 14:42:53.133694 30158 raft_consensus.cc:2454] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b46588679eb406796517b8da42aaa34 in term 0.
I20260812 14:42:53.133723 29619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "268ddca4079749edb6c282f4e1c91dec" candidate_uuid: "3b46588679eb406796517b8da42aaa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" is_pre_election: true
I20260812 14:42:53.133800 29619 raft_consensus.cc:2454] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b46588679eb406796517b8da42aaa34 in term 0.
I20260812 14:42:53.133896 29765 leader_election.cc:302] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b46588679eb406796517b8da42aaa34, 4f770c70c2ff459abcd13c5e844a07be; no voters:
I20260812 14:42:53.134060 30430 raft_consensus.cc:2781] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:53.134111 30430 raft_consensus.cc:482] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:53.134136 30430 raft_consensus.cc:3037] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:53.134812 30430 raft_consensus.cc:504] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.134923 30430 leader_election.cc:288] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [CANDIDATE]: Term 1 election: Requested vote from peers 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449), 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195:45467)
I20260812 14:42:53.135067 29619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "268ddca4079749edb6c282f4e1c91dec" candidate_uuid: "3b46588679eb406796517b8da42aaa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707d1c53fc2d42fd873b6f288c5ae04d"
I20260812 14:42:53.135102 30158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "268ddca4079749edb6c282f4e1c91dec" candidate_uuid: "3b46588679eb406796517b8da42aaa34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f770c70c2ff459abcd13c5e844a07be"
I20260812 14:42:53.135146 29619 raft_consensus.cc:3037] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:53.135162 30158 raft_consensus.cc:3037] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:53.135890 29619 raft_consensus.cc:2454] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b46588679eb406796517b8da42aaa34 in term 1.
I20260812 14:42:53.135952 30158 raft_consensus.cc:2454] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b46588679eb406796517b8da42aaa34 in term 1.
I20260812 14:42:53.136090 29765 leader_election.cc:302] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b46588679eb406796517b8da42aaa34, 4f770c70c2ff459abcd13c5e844a07be; no voters:
I20260812 14:42:53.136185 30430 raft_consensus.cc:2781] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:53.136330 30430 raft_consensus.cc:686] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 1 LEADER]: Becoming Leader. State: Replica: 3b46588679eb406796517b8da42aaa34, State: Running, Role: LEADER
I20260812 14:42:53.136390 30430 consensus_queue.cc:237] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } }
I20260812 14:42:53.136632 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.136506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:53563)
0812 14:42:53.136583 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.136610 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 14:42:53.137044 29376 catalog_manager.cc:5495] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b46588679eb406796517b8da42aaa34 (127.15.227.194). New cstate: current_term: 1 leader_uuid: "3b46588679eb406796517b8da42aaa34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b46588679eb406796517b8da42aaa34" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 42803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:53.148545 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.146326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46110 (local address 127.15.227.194:42803)
0812 14:42:53.146372 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:53.146374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:53.146401 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:42:53.146477 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:53.146478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:53.146495 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:53.146556 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:53.146689 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.147185 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.147883 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:53.148081 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:53.148142 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:53.148266 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:53.148361 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:53.148419 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:53.148466 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 14:42:53.148485 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:42:53.149466 30158 raft_consensus.cc:1261] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 1 FOLLOWER]: Refusing update from remote peer 3b46588679eb406796517b8da42aaa34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:53.149659 29619 raft_consensus.cc:1261] T 268ddca4079749edb6c282f4e1c91dec P 707d1c53fc2d42fd873b6f288c5ae04d [term 1 FOLLOWER]: Refusing update from remote peer 3b46588679eb406796517b8da42aaa34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:53.149724 30430 consensus_queue.cc:1035] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f770c70c2ff459abcd13c5e844a07be" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:53.149897 30444 consensus_queue.cc:1035] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "707d1c53fc2d42fd873b6f288c5ae04d" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:53.150605 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.150454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.193:46157)
0812 14:42:53.150540 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.150577 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 14:42:53.150674 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.150555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.195:36285)
0812 14:42:53.150622 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.150647 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 14:42:53.151748 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.151560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:57009)
0812 14:42:53.151696 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.151724 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260812 14:42:53.151988 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.151895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.193:49707)
0812 14:42:53.151933 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.151963 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260812 14:42:53.152681 30437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.152599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.195:51149)
0812 14:42:53.152631 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.152657 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 14:42:53.152711 30443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:53.152627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36191 (local address 127.15.227.194:41345)
0812 14:42:53.152660 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.152687 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:36191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260812 14:42:53.154341 29376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:53.154415 29376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:53.155681 29376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ebdf2c2baf74bfcba8af5bf8abef14f: Sending DeleteTablet for 3 replicas of tablet 268ddca4079749edb6c282f4e1c91dec
I20260812 14:42:53.155921 29856 tablet_service.cc:1514] Processing DeleteTablet for tablet 268ddca4079749edb6c282f4e1c91dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:53 UTC) from {username='oryx', principal='oryx/127.15.227.252@KRBTEST.COM'} at 127.0.0.1:46100
I20260812 14:42:53.155936 30120 tablet_service.cc:1514] Processing DeleteTablet for tablet 268ddca4079749edb6c282f4e1c91dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:53 UTC) from {username='oryx', principal='oryx/127.15.227.252@KRBTEST.COM'} at 127.0.0.1:56122
I20260812 14:42:53.155969 29579 tablet_service.cc:1514] Processing DeleteTablet for tablet 268ddca4079749edb6c282f4e1c91dec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:53 UTC) from {username='oryx', principal='oryx/127.15.227.252@KRBTEST.COM'} at 127.0.0.1:46696
I20260812 14:42:53.156140 30469 tablet_replica.cc:330] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: stopping tablet replica
I20260812 14:42:53.156199 30469 raft_consensus.cc:2229] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:53.156284 30469 raft_consensus.cc:2258] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:53.156378 30472 tablet_replica.cc:330] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: stopping tablet replica
I20260812 14:42:53.156428 30472 raft_consensus.cc:2229] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:53.156468 30472 raft_consensus.cc:2258] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:53.156487 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 29310
I20260812 14:42:53.159576 30469 ts_tablet_manager.cc:1905] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:53.161829 30469 ts_tablet_manager.cc:1918] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:53.161940 30469 log.cc:1196] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/268ddca4079749edb6c282f4e1c91dec
I20260812 14:42:53.162221 30469 ts_tablet_manager.cc:1939] T 268ddca4079749edb6c282f4e1c91dec P 3b46588679eb406796517b8da42aaa34: Deleting consensus metadata
I20260812 14:42:53.162523 29354 catalog_manager.cc:4841] TS 3b46588679eb406796517b8da42aaa34 (127.15.227.194:42803): tablet 268ddca4079749edb6c282f4e1c91dec (table test-table [id=ebf890a9c49b4019a75af9c49c522e1f]) successfully deleted
I20260812 14:42:53.168587 30472 ts_tablet_manager.cc:1905] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:53.170217 30472 ts_tablet_manager.cc:1918] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:53.170284 30472 log.cc:1196] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/268ddca4079749edb6c282f4e1c91dec
I20260812 14:42:53.170526 30472 ts_tablet_manager.cc:1939] T 268ddca4079749edb6c282f4e1c91dec P 4f770c70c2ff459abcd13c5e844a07be: Deleting consensus metadata
I20260812 14:42:53.170867 29356 catalog_manager.cc:4841] TS 4f770c70c2ff459abcd13c5e844a07be (127.15.227.195:45467): tablet 268ddca4079749edb6c282f4e1c91dec (table test-table [id=ebf890a9c49b4019a75af9c49c522e1f]) successfully deleted
W20260812 14:42:53.173542 30415 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:53.173401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44449 (local address 127.0.0.1:46708)
0812 14:42:53.173458 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:53.173486 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:44449: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 14:42:53.173678 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 29728
W20260812 14:42:53.182880 29354 catalog_manager.cc:4584] TS 707d1c53fc2d42fd873b6f288c5ae04d (127.15.227.193:44449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 268ddca4079749edb6c282f4e1c91dec: Network error: Client connection negotiation failed: client connection to 127.15.227.193:44449: connect: Connection reset by peer (error 104)
I20260812 14:42:53.191237 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 29998
I20260812 14:42:53.205041 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 28806
I20260812 14:42:53.213389 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 29145
2026-08-12T14:42:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4734 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:53 dist-test-slave-q9h9 krb5kdc[30484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:53 dist-test-slave-q9h9 krb5kdc[30484](info): set up 2 sockets
Aug 12 14:42:53 dist-test-slave-q9h9 krb5kdc[30484](info): commencing operation
krb5kdc: starting...
W20260812 14:42:54.410364 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:54 dist-test-slave-q9h9 krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab.
Entry for principal oryx/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab.
I20260812 14:42:54.520164 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33953
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab
--principal=oryx/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:54.598716 30517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:54.598906 30517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:54.598935 30517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:54.600032 30517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:54.600076 30517 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 14:42:54.600093 30517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:54.600116 30517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:54.600134 30517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:54.600150 30517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:54.601619 30517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33953
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.254.keytab
--principal=oryx/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:54.601873 30517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:54.602079 30517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:54.606503 30528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:54.606513 30524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:54.606640 30517 server_base.cc:1034] running on GCE node
W20260812 14:42:54.606520 30526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:54.606890 30517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:54.607146 30517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:54.608279 30517 hybrid_clock.cc:648] HybridClock initialized: now 1786545774608263 us; error 27 us; skew 500 ppm
Aug 12 14:42:54 dist-test-slave-q9h9 krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:54.610518 30517 init.cc:377] Logged in from keytab as oryx/127.15.227.254@KRBTEST.COM (short username oryx)
I20260812 14:42:54.610900 30517 webserver.cc:460] Webserver started at http://127.15.227.254:34435/ using document root <none> and password file <none>
I20260812 14:42:54.611090 30517 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:54.611155 30517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:54.611274 30517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:54.612223 30517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "b3dde5f6d85c4be09869d19092f73630"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "84c92c4634a7525f3154b6ced735dcc2"
server_key_iv: "540517feff14ca5a80d956af72b9dfe4"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.612534 30517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "b3dde5f6d85c4be09869d19092f73630"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "84c92c4634a7525f3154b6ced735dcc2"
server_key_iv: "540517feff14ca5a80d956af72b9dfe4"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.613698 30517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:42:54.614346 30539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:54.614486 30517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:54.614565 30517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "b3dde5f6d85c4be09869d19092f73630"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "84c92c4634a7525f3154b6ced735dcc2"
server_key_iv: "540517feff14ca5a80d956af72b9dfe4"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.614621 30517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:54.623441 30517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:54.625109 30517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:54.625236 30517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:54.629669 30517 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:33953
I20260812 14:42:54.629727 30661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:33953 every 8 connection(s)
I20260812 14:42:54.630000 30517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:54.630642 30663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:54.632711 30663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Bootstrap starting.
I20260812 14:42:54.633270 30663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:54.633534 30663 log.cc:824] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:54.634106 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 30517
I20260812 14:42:54.634178 30663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: No bootstrap required, opened a new log
I20260812 14:42:54.634202 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:54.634382 16271 external_mini_cluster.cc:1428] Setting key aee3066c1e8d78751b7e9ce4fd1ff6e8
I20260812 14:42:54.636397 30663 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3dde5f6d85c4be09869d19092f73630" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33953 } }
I20260812 14:42:54.636523 30663 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:54.636556 30663 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dde5f6d85c4be09869d19092f73630, State: Initialized, Role: FOLLOWER
I20260812 14:42:54.636710 30663 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3dde5f6d85c4be09869d19092f73630" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33953 } }
I20260812 14:42:54.636794 30663 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:54.636831 30663 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:54.636869 30663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:54.637418 30663 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3dde5f6d85c4be09869d19092f73630" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33953 } }
I20260812 14:42:54.637526 30663 leader_election.cc:302] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3dde5f6d85c4be09869d19092f73630; no voters:
I20260812 14:42:54.637679 30663 leader_election.cc:288] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:54.637709 30670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:54.637840 30670 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [term 1 LEADER]: Becoming Leader. State: Replica: b3dde5f6d85c4be09869d19092f73630, State: Running, Role: LEADER
I20260812 14:42:54.637917 30663 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:54.637933 30670 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3dde5f6d85c4be09869d19092f73630" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33953 } }
I20260812 14:42:54.638300 30670 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3dde5f6d85c4be09869d19092f73630. Latest consensus state: current_term: 1 leader_uuid: "b3dde5f6d85c4be09869d19092f73630" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3dde5f6d85c4be09869d19092f73630" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33953 } } }
I20260812 14:42:54.638386 30670 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:54.638311 30673 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3dde5f6d85c4be09869d19092f73630" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3dde5f6d85c4be09869d19092f73630" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33953 } } }
I20260812 14:42:54.638535 30673 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630 [sys.catalog]: This master's current role is: LEADER
W20260812 14:42:54.639719 30702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:42:54.639782 30702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:42:54.639837 30679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:54.639984 30679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:54.641688 30679 catalog_manager.cc:1347] Generated new cluster ID: 1ddd9f96068e4703afafd5139fc65f69
I20260812 14:42:54.641742 30679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:42:54 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:54.650475 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.634836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.15.227.254:33953)
0812 14:42:54.634989 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:54.634997 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:54.636752 (+ 1755us) server_negotiation.cc:407] Connection header received
0812 14:42:54.636920 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:54.636926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:54.636968 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:54.637073 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:54.637870 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.639258 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.640311 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.640821 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.640929 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.647592 (+ 6663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:54.647595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:54.647595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:54.647605 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:54.649051 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.649367 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.649370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.649370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.649449 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.649621 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.649621 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.649622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.649855 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:54.650000 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:54.650133 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 14:42:54.650279 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
I20260812 14:42:54.670104 30679 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:54.670568 30679 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:54.673161 30679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Generated new TSK 0
I20260812 14:42:54.673308 30679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab.
Entry for principal oryx/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab.
I20260812 14:42:54.700626 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33953
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab
--principal=oryx/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:54.785802 30711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:54.785984 30711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:54.786007 30711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:54.787139 30711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:54.787176 30711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:54.787195 30711 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:54.787211 30711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:54.788615 30711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab
--principal=oryx/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:54.788781 30711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:54.788941 30711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:54.793617 30723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:54.793676 30721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:54.793617 30729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:54.793872 30711 server_base.cc:1034] running on GCE node
I20260812 14:42:54.794159 30711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:54.794399 30711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:54.795543 30711 hybrid_clock.cc:648] HybridClock initialized: now 1786545774795525 us; error 32 us; skew 500 ppm
Aug 12 14:42:54 dist-test-slave-q9h9 krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:54.797814 30711 init.cc:377] Logged in from keytab as oryx/127.15.227.193@KRBTEST.COM (short username oryx)
I20260812 14:42:54.798266 30711 webserver.cc:460] Webserver started at http://127.15.227.193:46089/ using document root <none> and password file <none>
I20260812 14:42:54.798460 30711 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:54.798506 30711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:54.798630 30711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:54.799578 30711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "5a25b1b48ac743639002c30cc6ac684a"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "1d3f0c68e4d27ce6a4cb2702725093ae"
server_key_iv: "25eddb3738c953d5bc5661822ff6325f"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.799901 30711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "5a25b1b48ac743639002c30cc6ac684a"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "1d3f0c68e4d27ce6a4cb2702725093ae"
server_key_iv: "25eddb3738c953d5bc5661822ff6325f"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.801093 30711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:54.801702 30737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:54.801848 30711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:54.801929 30711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "5a25b1b48ac743639002c30cc6ac684a"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "1d3f0c68e4d27ce6a4cb2702725093ae"
server_key_iv: "25eddb3738c953d5bc5661822ff6325f"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.801995 30711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:54.813840 30711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:54.815407 30711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:54.815531 30711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:54.815979 30711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:54.816017 30711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:54.816043 30711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:54.816061 30711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:54.822273 30711 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:36437
I20260812 14:42:54.822329 30965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:36437 every 8 connection(s)
Aug 12 14:42:54 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:54.822629 30711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:54.825215 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.816390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:58391 (local address 127.15.227.254:33953)
0812 14:42:54.816447 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:54.816449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:54.816594 (+ 145us) server_negotiation.cc:407] Connection header received
0812 14:42:54.817071 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:54.817073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:54.817086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:54.817155 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:54.817889 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.818675 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.820106 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.820436 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.820504 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.822894 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:54.822896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:54.822896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:54.822906 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:54.824207 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.824557 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.824558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.824558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.824602 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.824764 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.824764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.824764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.824854 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:54.824902 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:54.825088 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 14:42:54.825153 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:42:54.825374 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.816339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33953 (local address 127.15.227.193:58391)
0812 14:42:54.816480 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:54.816515 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:54.816987 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:54.817209 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:54.817214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:54.817522 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:54.817792 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.817798 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.818775 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.818776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.820015 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.820019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.820526 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.820527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.820646 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.821083 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:54.821093 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:54.822786 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:54.824282 (+ 1496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.824290 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.824292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.824499 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.824639 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.824640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.824640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.824721 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.824900 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:54.824902 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:54.825000 (+ 98us) client_negotiation.cc:769] Sending connection context
0812 14:42:54.825076 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:42:54.825185 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260812 14:42:54.825811 30968 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33953
I20260812 14:42:54.825887 30968 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:54.826092 30968 heartbeater.cc:511] Master 127.15.227.254:33953 requested a full tablet report, sending...
I20260812 14:42:54.826122 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 30711
I20260812 14:42:54.826225 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:54.826442 16271 external_mini_cluster.cc:1428] Setting key 37152642cef856cc8ee10d28587ab984
I20260812 14:42:54.826720 30564 ts_manager.cc:194] Registered new tserver with Master: 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437)
I20260812 14:42:54.827328 30564 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:58391
I20260812 14:42:54.832857 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.827608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33953 (local address 127.15.227.193:58199)
0812 14:42:54.827776 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:54.827785 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:54.827845 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:54.827953 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:54.827954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:54.828148 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:54.828219 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.828221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.828947 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.828948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.829430 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.829431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.829838 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.829838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.829881 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.830246 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:54.830250 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:54.830514 (+ 264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:54.832004 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.832005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.832005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.832166 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.832358 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.832358 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.832358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.832406 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.832632 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:54.832633 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:54.832680 (+ 47us) client_negotiation.cc:769] Sending connection context
0812 14:42:54.832725 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:42:54.832793 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260812 14:42:54.832911 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.827664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:58199 (local address 127.15.227.254:33953)
0812 14:42:54.827710 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:54.827712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:54.827847 (+ 135us) server_negotiation.cc:407] Connection header received
0812 14:42:54.827889 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:54.827890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:54.827901 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:54.827940 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:54.828278 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.828841 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.829503 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.829772 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.829831 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.830592 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:54.830594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:54.830594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:54.830601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:54.831932 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.832231 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.832232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.832232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.832299 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.832455 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.832455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.832456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.832578 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:54.832645 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:54.832752 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 14:42:54.832826 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for oryx/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab.
Entry for principal oryx/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab.
I20260812 14:42:54.849972 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33953
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab
--principal=oryx/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:54.928761 30995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:54.928944 30995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:54.928974 30995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:54.930119 30995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:54.930164 30995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:54.930191 30995 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:54.930215 30995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:54.931676 30995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab
--principal=oryx/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:54.931890 30995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:54.932088 30995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:54.936792 31001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:54.936779 31005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:54.936817 31002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:54.936825 30995 server_base.cc:1034] running on GCE node
I20260812 14:42:54.937197 30995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:54.937438 30995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:54.938566 30995 hybrid_clock.cc:648] HybridClock initialized: now 1786545774938552 us; error 27 us; skew 500 ppm
Aug 12 14:42:54 dist-test-slave-q9h9 krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:54.940940 30995 init.cc:377] Logged in from keytab as oryx/127.15.227.194@KRBTEST.COM (short username oryx)
I20260812 14:42:54.941324 30995 webserver.cc:460] Webserver started at http://127.15.227.194:46873/ using document root <none> and password file <none>
I20260812 14:42:54.941505 30995 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:54.941551 30995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:54.941672 30995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:54.942641 30995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "aca95d5482be426f8b28ba89e76e508e"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "6ca44a86d46ad13e0977f2d2cc060fb7"
server_key_iv: "9804d652c95954c2dde4f76c2ec47568"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.942955 30995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "aca95d5482be426f8b28ba89e76e508e"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "6ca44a86d46ad13e0977f2d2cc060fb7"
server_key_iv: "9804d652c95954c2dde4f76c2ec47568"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.944128 30995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:54.944803 31018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:54.944964 30995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:54.945042 30995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "aca95d5482be426f8b28ba89e76e508e"
format_stamp: "Formatted at 2026-08-12 14:42:54 on dist-test-slave-q9h9"
server_key: "6ca44a86d46ad13e0977f2d2cc060fb7"
server_key_iv: "9804d652c95954c2dde4f76c2ec47568"
server_key_version: "encryptionkey@0"
I20260812 14:42:54.945099 30995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:54.955435 30995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:54.957145 30995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:54.957283 30995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:54.957796 30995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:54.957845 30995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:54.957871 30995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:54.957908 30995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 14:42:54 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:54.964521 30995 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:44735
I20260812 14:42:54.964576 31263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:44735 every 8 connection(s)
I20260812 14:42:54.964846 30995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:54.966661 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.958159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:56759 (local address 127.15.227.254:33953)
0812 14:42:54.958219 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:54.958222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:54.958490 (+ 268us) server_negotiation.cc:407] Connection header received
0812 14:42:54.958907 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:54.958908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:54.958923 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:54.958998 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:54.959600 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.960111 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.961070 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.961275 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.961331 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.964114 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:54.964115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:54.964116 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:54.964123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:54.965502 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.965882 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.965883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.965883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.965937 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.966164 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.966164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.966165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.966266 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:54.966317 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:54.966538 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 14:42:54.966595 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:42:54.966811 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.958203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33953 (local address 127.15.227.194:56759)
0812 14:42:54.958349 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:54.958386 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:54.958822 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:54.959050 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:54.959054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:54.959331 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:54.959506 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.959511 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.960200 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.960202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.960984 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.960986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.961352 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.961353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.961486 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.961947 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:54.961955 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:54.964036 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:54.965565 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.965569 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.965571 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.965818 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.965980 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.965980 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.965981 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.966112 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.966326 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:54.966330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:54.966437 (+ 107us) client_negotiation.cc:769] Sending connection context
0812 14:42:54.966514 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:42:54.966623 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260812 14:42:54.967260 31264 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33953
I20260812 14:42:54.967342 31264 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:54.967512 31264 heartbeater.cc:511] Master 127.15.227.254:33953 requested a full tablet report, sending...
I20260812 14:42:54.967901 30566 ts_manager.cc:194] Registered new tserver with Master: aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735)
I20260812 14:42:54.968360 30566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:56759
I20260812 14:42:54.972664 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.968248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:36645 (local address 127.15.227.254:33953)
0812 14:42:54.968303 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:54.968305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:54.968360 (+ 55us) server_negotiation.cc:407] Connection header received
0812 14:42:54.968402 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:54.968403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:54.968411 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:54.968468 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:54.968764 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.969197 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.969807 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.969969 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.970008 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.970682 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:54.970683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:54.970683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:54.970688 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:54.971846 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.972130 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.972130 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.972131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.972189 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:54.972311 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:54.972311 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:54.972312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:54.972423 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:54.972468 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:54.972560 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 14:42:54.972609 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:42:54.972668 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:54.968193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33953 (local address 127.15.227.194:36645)
0812 14:42:54.968308 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:54.968317 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:54.968367 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:54.968475 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:54.968476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:54.968629 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:54.968695 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.968696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.969279 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.969280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.969753 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:54.969754 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:54.970032 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:54.970033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:54.970097 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:54.970386 (+ 289us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:54.970389 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:54.970621 (+ 232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:54.971929 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.971930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.971930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.972076 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.972224 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:54.972224 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:54.972225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:54.972274 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:54.972465 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:54.972466 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:54.972498 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 14:42:54.972537 (+ 39us) client_negotiation.cc:240] Negotiation successful
0812 14:42:54.972600 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
I20260812 14:42:54.974072 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 30995
I20260812 14:42:54.974184 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:54.974371 16271 external_mini_cluster.cc:1428] Setting key 468e60acfe40fb14235dd8f8e62c259d
WARNING: no policy specified for oryx/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab.
Entry for principal oryx/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab.
I20260812 14:42:54.997921 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33953
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab
--principal=oryx/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:46215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:55.077900 31287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:55.078111 31287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:55.078143 31287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:55.079232 31287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:55.079276 31287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:55.079303 31287 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 14:42:55.079327 31287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:55.080801 31287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:46215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab
--principal=oryx/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.227.254:33953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:55.081020 31287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:55.081211 31287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:55.085958 31302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:55.085983 31287 server_base.cc:1034] running on GCE node
W20260812 14:42:55.085948 31299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:55.085932 31296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:55.086265 31287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:55.086481 31287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:55.087615 31287 hybrid_clock.cc:648] HybridClock initialized: now 1786545775087600 us; error 27 us; skew 500 ppm
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545775, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:55.089901 31287 init.cc:377] Logged in from keytab as oryx/127.15.227.195@KRBTEST.COM (short username oryx)
I20260812 14:42:55.090327 31287 webserver.cc:460] Webserver started at http://127.15.227.195:43837/ using document root <none> and password file <none>
I20260812 14:42:55.090521 31287 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:55.090567 31287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:55.090692 31287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:55.091643 31287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2"
format_stamp: "Formatted at 2026-08-12 14:42:55 on dist-test-slave-q9h9"
server_key: "a409f160e07be6cb77baf5b809956d2b"
server_key_iv: "60cc3ecad67b9de26bc136579c277f8c"
server_key_version: "encryptionkey@0"
I20260812 14:42:55.091966 31287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2"
format_stamp: "Formatted at 2026-08-12 14:42:55 on dist-test-slave-q9h9"
server_key: "a409f160e07be6cb77baf5b809956d2b"
server_key_iv: "60cc3ecad67b9de26bc136579c277f8c"
server_key_version: "encryptionkey@0"
I20260812 14:42:55.093123 31287 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:42:55.093755 31311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.093902 31287 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:55.093981 31287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2"
format_stamp: "Formatted at 2026-08-12 14:42:55 on dist-test-slave-q9h9"
server_key: "a409f160e07be6cb77baf5b809956d2b"
server_key_iv: "60cc3ecad67b9de26bc136579c277f8c"
server_key_version: "encryptionkey@0"
I20260812 14:42:55.094066 31287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:55.101855 31287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:55.103935 31287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:55.104064 31287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:55.104547 31287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:55.104604 31287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.104650 31287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:55.104674 31287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.110801 31287 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:41843
I20260812 14:42:55.110924 31522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:41843 every 8 connection(s)
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545775, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:55.111164 31287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:42:55.112218 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 31287
I20260812 14:42:55.112325 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:55.112532 16271 external_mini_cluster.cc:1428] Setting key 8e23db4aca51cce15d90df9223bf4701
I20260812 14:42:55.114125 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.104955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:38041 (local address 127.15.227.254:33953)
0812 14:42:55.105020 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.105023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.105159 (+ 136us) server_negotiation.cc:407] Connection header received
0812 14:42:55.105770 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.105772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.105788 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.105842 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:55.106581 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.107313 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.108778 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.109066 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.109153 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.111528 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:55.111530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:55.111530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:55.111537 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:55.112929 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.113323 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.113324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.113324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.113388 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.113595 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.113595 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.113595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.113703 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:55.113749 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.113933 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.113998 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:42:55.114239 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.104917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33953 (local address 127.15.227.195:38041)
0812 14:42:55.105047 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.105083 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.105641 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.105890 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.105893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.106230 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:55.106478 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.106484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.107426 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.107427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.108686 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.108689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.109154 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.109155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.109285 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.109743 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:55.109752 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:55.111423 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:55.112995 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:55.113001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:55.113003 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:55.113266 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:55.113448 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:55.113449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:55.113449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:55.113552 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:55.113748 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:55.113751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:55.113848 (+ 97us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.113918 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.114056 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260812 14:42:55.114652 31523 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33953
I20260812 14:42:55.114729 31523 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:55.114893 31523 heartbeater.cc:511] Master 127.15.227.254:33953 requested a full tablet report, sending...
I20260812 14:42:55.115213 30566 ts_manager.cc:194] Registered new tserver with Master: bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843)
I20260812 14:42:55.115577 30566 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:38041
I20260812 14:42:55.119952 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.115591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:33157 (local address 127.15.227.254:33953)
0812 14:42:55.115746 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.115748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.115754 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:42:55.115770 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.115771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.115780 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.115832 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:55.116162 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.116639 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.117266 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.117431 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.117484 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.118223 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:55.118225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:55.118225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:55.118231 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:55.119223 (+ 992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.119485 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.119486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.119486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.119528 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.119636 (+ 108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.119636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.119636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.119722 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:55.119763 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.119849 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.119899 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260812 14:42:55.119963 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.115526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33953 (local address 127.15.227.195:33157)
0812 14:42:55.115635 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.115645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.115699 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.115836 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.115836 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.116019 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:55.116088 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.116090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.116705 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.116705 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.117210 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.117212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.117482 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.117483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.117524 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.117836 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:55.117840 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:55.118120 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:55.119285 (+ 1165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:55.119286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:55.119286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:55.119432 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:55.119564 (+ 132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:55.119564 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:55.119565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:55.119605 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:55.119761 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:55.119761 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:55.119792 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.119834 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.119898 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260812 14:42:55.124064 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545775, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:42:55.145321 30669 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:55.140575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.15.227.254:33953)
0812 14:42:55.140630 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.140632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.140640 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:55.140677 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.140678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.140686 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.140730 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:55.141238 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.141701 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.142415 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.142593 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.142640 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.145238 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59060: BlockingRecv error: recv got EOF from 127.0.0.1:59060 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545775, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:55.151925 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.146073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.15.227.254:33953)
0812 14:42:55.146126 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.146128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.146196 (+ 68us) server_negotiation.cc:407] Connection header received
0812 14:42:55.146263 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.146264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.146273 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.146317 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:55.146720 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.147159 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.147816 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.147992 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.148042 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.149838 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:55.149839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:55.149839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:55.149847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:55.150998 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.151282 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.151283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.151284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.151333 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.151494 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.151495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.151495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.151591 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:55.151674 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.151774 (+ 100us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.151844 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:42:55.153236 30566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:55.154842 30566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:55.160530 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.156721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:41843 (local address 127.0.0.1:37384)
0812 14:42:55.156874 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.156911 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.156969 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.157410 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.157411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.157413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.157453 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.157454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.158688 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.158689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.159497 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.159499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.160324 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.160325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.160383 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.160394 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.160446 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.160481 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":54,"threads_started":1}
I20260812 14:42:55.160609 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.156818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.15.227.195:41843)
0812 14:42:55.157260 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.157264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.157275 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:55.157310 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.157312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.157327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.157402 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.157533 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.158631 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.159578 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.160234 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.160334 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.160390 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.160471 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.160509 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":83,"threads_started":1}
I20260812 14:42:55.161024 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.156721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:44735 (local address 127.0.0.1:40792)
0812 14:42:55.156988 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.156996 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.157031 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.157287 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.157290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.157293 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.157352 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.157353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.158623 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.158624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.159736 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.159739 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.160599 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.160600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.160881 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.160886 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.160921 (+ 35us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.160959 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":31,"spinlock_wait_cycles":768,"thread_start_us":63,"threads_started":1}
I20260812 14:42:55.161276 31403 tablet_service.cc:1467] Processing CreateTablet for tablet 27e299ba118b4badbd0731599bb30134 (DEFAULT_TABLE table=test-table [id=87a7939de5264d12b31d4a25a15e1559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.161615 31575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.156912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36437 (local address 127.0.0.1:32884)
0812 14:42:55.157102 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.157109 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.157133 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.157499 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.157500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.157503 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.157547 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.157549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.158488 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.158489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.159752 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.159754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.161396 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.161397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.161455 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.161460 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.161511 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.161549 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":22,"thread_start_us":36,"threads_started":1}
I20260812 14:42:55.161648 31403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e299ba118b4badbd0731599bb30134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.161819 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.156835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40792 (local address 127.15.227.194:44735)
0812 14:42:55.157112 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.157116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.157129 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:42:55.157168 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.157171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.157186 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.157268 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.157428 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.158546 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.159824 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.160518 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.161498 (+ 980us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.161550 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.161571 (+ 21us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.161603 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":77,"threads_started":1}
I20260812 14:42:55.162113 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.157068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32884 (local address 127.15.227.193:36437)
0812 14:42:55.157261 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.157265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.157274 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:55.157311 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.157312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.157327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.157404 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.157628 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.158412 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.160720 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.161319 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.161395 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.161429 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.161888 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.161938 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260812 14:42:55.162458 31121 tablet_service.cc:1467] Processing CreateTablet for tablet 27e299ba118b4badbd0731599bb30134 (DEFAULT_TABLE table=test-table [id=87a7939de5264d12b31d4a25a15e1559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.162802 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e299ba118b4badbd0731599bb30134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.162712 30839 tablet_service.cc:1467] Processing CreateTablet for tablet 27e299ba118b4badbd0731599bb30134 (DEFAULT_TABLE table=test-table [id=87a7939de5264d12b31d4a25a15e1559]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.163194 30839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 27e299ba118b4badbd0731599bb30134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.164582 31578 tablet_bootstrap.cc:492] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Bootstrap starting.
I20260812 14:42:55.165122 31578 tablet_bootstrap.cc:654] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.165339 31578 log.cc:824] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:55.165925 31578 tablet_bootstrap.cc:492] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: No bootstrap required, opened a new log
I20260812 14:42:55.165967 31578 ts_tablet_manager.cc:1397] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:55.166558 31581 tablet_bootstrap.cc:492] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Bootstrap starting.
I20260812 14:42:55.166857 31584 tablet_bootstrap.cc:492] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Bootstrap starting.
I20260812 14:42:55.167169 31581 tablet_bootstrap.cc:654] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.167477 31584 tablet_bootstrap.cc:654] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.167471 31581 log.cc:824] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:55.167829 31584 log.cc:824] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:55.167917 31578 raft_consensus.cc:348] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.168084 31578 raft_consensus.cc:374] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.168128 31578 raft_consensus.cc:729] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5cb51c9c5f4e95b8c8d19c9c31b2e2, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.168260 31581 tablet_bootstrap.cc:492] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: No bootstrap required, opened a new log
I20260812 14:42:55.168326 31581 ts_tablet_manager.cc:1397] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:55.168327 31578 consensus_queue.cc:260] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.168553 31584 tablet_bootstrap.cc:492] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: No bootstrap required, opened a new log
I20260812 14:42:55.168587 31578 ts_tablet_manager.cc:1428] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 14:42:55.168615 31584 ts_tablet_manager.cc:1397] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:42:55.168614 31523 heartbeater.cc:503] Master 127.15.227.254:33953 was elected leader, sending a full tablet report...
I20260812 14:42:55.169910 31581 raft_consensus.cc:348] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.170047 31581 raft_consensus.cc:374] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.170087 31581 raft_consensus.cc:729] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.170177 31581 consensus_queue.cc:260] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.170404 31581 ts_tablet_manager.cc:1428] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:42:55.170433 31264 heartbeater.cc:503] Master 127.15.227.254:33953 was elected leader, sending a full tablet report...
I20260812 14:42:55.170428 31584 raft_consensus.cc:348] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.170578 31584 raft_consensus.cc:374] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.170622 31584 raft_consensus.cc:729] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25b1b48ac743639002c30cc6ac684a, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.170727 31584 consensus_queue.cc:260] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.170987 31584 ts_tablet_manager.cc:1428] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:42:55.171002 30968 heartbeater.cc:503] Master 127.15.227.254:33953 was elected leader, sending a full tablet report...
I20260812 14:42:55.177412 31593 raft_consensus.cc:482] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:55.177476 31593 raft_consensus.cc:504] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.177736 31593 leader_election.cc:288] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843), 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437)
I20260812 14:42:55.180337 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.177934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:55973 (local address 127.15.227.195:41843)
0812 14:42:55.177981 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.177983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.177993 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:42:55.178098 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.178099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.178110 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.178154 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.178293 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.178724 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.179595 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.179999 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.180060 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.180069 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.180254 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.180285 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:42:55.180406 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.178113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:57661 (local address 127.15.227.193:36437)
0812 14:42:55.178161 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.178162 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.178169 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:55.178220 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.178221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.178229 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.178280 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.178405 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.178830 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.179694 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.180118 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.180179 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.180187 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.180321 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.180350 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:42:55.180436 31597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.177975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36437 (local address 127.15.227.194:57661)
0812 14:42:55.178113 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.178123 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.178174 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.178282 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.178283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.178284 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.178355 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.178357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.178895 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.178895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.179612 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.179614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.180193 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.180194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.180267 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.180270 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.180311 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.180346 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260812 14:42:55.180353 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.177888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:41843 (local address 127.15.227.194:55973)
0812 14:42:55.177943 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.177953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.177991 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.178165 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.178166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.178167 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.178224 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.178225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.178794 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.178795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.179520 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.179522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.180099 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.180100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.180186 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.180189 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.180233 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.180269 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":16}
I20260812 14:42:55.180709 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e299ba118b4badbd0731599bb30134" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a" is_pre_election: true
I20260812 14:42:55.180815 30875 raft_consensus.cc:2454] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 0.
I20260812 14:42:55.180948 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e299ba118b4badbd0731599bb30134" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" is_pre_election: true
I20260812 14:42:55.180996 31019 leader_election.cc:302] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a25b1b48ac743639002c30cc6ac684a, aca95d5482be426f8b28ba89e76e508e; no voters:
I20260812 14:42:55.181039 31448 raft_consensus.cc:2454] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 0.
I20260812 14:42:55.181124 31593 raft_consensus.cc:2781] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:55.181167 31593 raft_consensus.cc:482] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:55.181186 31593 raft_consensus.cc:3037] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.181922 31593 raft_consensus.cc:504] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.182044 31593 leader_election.cc:288] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 election: Requested vote from peers bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843), 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437)
I20260812 14:42:55.182205 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e299ba118b4badbd0731599bb30134" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2"
I20260812 14:42:55.182219 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "27e299ba118b4badbd0731599bb30134" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a"
I20260812 14:42:55.182281 30875 raft_consensus.cc:3037] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.182276 31448 raft_consensus.cc:3037] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.182862 31448 raft_consensus.cc:2454] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 1.
I20260812 14:42:55.182871 30875 raft_consensus.cc:2454] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 1.
I20260812 14:42:55.183017 31019 leader_election.cc:302] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aca95d5482be426f8b28ba89e76e508e, bc5cb51c9c5f4e95b8c8d19c9c31b2e2; no voters:
I20260812 14:42:55.183106 31593 raft_consensus.cc:2781] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:55.183250 31593 raft_consensus.cc:686] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 1 LEADER]: Becoming Leader. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Running, Role: LEADER
I20260812 14:42:55.183350 31593 consensus_queue.cc:237] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } }
I20260812 14:42:55.184019 30566 catalog_manager.cc:5495] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e reported cstate change: term changed from 0 to 1, leader changed from <none> to aca95d5482be426f8b28ba89e76e508e (127.15.227.194). New cstate: current_term: 1 leader_uuid: "aca95d5482be426f8b28ba89e76e508e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:55.189743 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.187689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40802 (local address 127.15.227.194:44735)
0812 14:42:55.187740 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.187742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.187750 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:55.187795 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.187796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.187811 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.187867 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.187976 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.188403 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.189087 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.189279 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.189330 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.189473 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.189555 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.189618 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.189668 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.189684 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:42:55.191260 31448 raft_consensus.cc:1261] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Refusing update from remote peer aca95d5482be426f8b28ba89e76e508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.191280 30875 raft_consensus.cc:1261] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Refusing update from remote peer aca95d5482be426f8b28ba89e76e508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.191548 31599 consensus_queue.cc:1035] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.191670 31593 consensus_queue.cc:1035] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.194267 31611 mvcc.cc:205] Tried to move back new op lower bound from 7317691495181320192 to 7317691495151570944. Current Snapshot: MvccSnapshot[applied={T|T < 7317691495181320192}]
I20260812 14:42:55.196151 30564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.196224 30564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.196890 30564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Sending DeleteTablet for 3 replicas of tablet 27e299ba118b4badbd0731599bb30134
I20260812 14:42:55.197193 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e299ba118b4badbd0731599bb30134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:40792
I20260812 14:42:55.197220 30839 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e299ba118b4badbd0731599bb30134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:32884
I20260812 14:42:55.197221 31403 tablet_service.cc:1514] Processing DeleteTablet for tablet 27e299ba118b4badbd0731599bb30134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:37384
I20260812 14:42:55.197521 31637 tablet_replica.cc:330] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: stopping tablet replica
I20260812 14:42:55.197623 31637 raft_consensus.cc:2229] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.197624 30564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260812 14:42:55.197724 31637 raft_consensus.cc:2258] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260812 14:42:55.197767 30564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:55.197996 31641 tablet_replica.cc:330] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: stopping tablet replica
I20260812 14:42:55.198148 31641 raft_consensus.cc:2229] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.198316 31641 raft_consensus.cc:2258] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.198514 31636 tablet_replica.cc:330] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: stopping tablet replica
I20260812 14:42:55.198643 31636 raft_consensus.cc:2229] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:55.198836 31636 raft_consensus.cc:2258] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.199011 31403 tablet_service.cc:1467] Processing CreateTablet for tablet c3f08b16304648e3ab795aa62b2c3382 (DEFAULT_TABLE table=test-table [id=d49ef647062543ec8cc53b7f9ea2adf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.199005 30839 tablet_service.cc:1467] Processing CreateTablet for tablet c3f08b16304648e3ab795aa62b2c3382 (DEFAULT_TABLE table=test-table [id=d49ef647062543ec8cc53b7f9ea2adf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.199106 31403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3f08b16304648e3ab795aa62b2c3382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.199106 30839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3f08b16304648e3ab795aa62b2c3382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.200289 31584 tablet_bootstrap.cc:492] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Bootstrap starting.
I20260812 14:42:55.200671 31584 tablet_bootstrap.cc:654] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.200731 31578 tablet_bootstrap.cc:492] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Bootstrap starting.
I20260812 14:42:55.198938 31121 tablet_service.cc:1467] Processing CreateTablet for tablet c3f08b16304648e3ab795aa62b2c3382 (DEFAULT_TABLE table=test-table [id=d49ef647062543ec8cc53b7f9ea2adf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.201126 31578 tablet_bootstrap.cc:654] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.201181 31641 ts_tablet_manager.cc:1905] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.201224 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c3f08b16304648e3ab795aa62b2c3382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.202440 31581 tablet_bootstrap.cc:492] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Bootstrap starting.
I20260812 14:42:55.202505 31641 ts_tablet_manager.cc:1918] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:55.202571 31641 log.cc:1196] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/27e299ba118b4badbd0731599bb30134
I20260812 14:42:55.202829 31641 ts_tablet_manager.cc:1939] T 27e299ba118b4badbd0731599bb30134 P 5a25b1b48ac743639002c30cc6ac684a: Deleting consensus metadata
I20260812 14:42:55.202836 31581 tablet_bootstrap.cc:654] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.203230 30540 catalog_manager.cc:4841] TS 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437): tablet 27e299ba118b4badbd0731599bb30134 (table test-table [id=87a7939de5264d12b31d4a25a15e1559]) successfully deleted
I20260812 14:42:55.203478 31584 tablet_bootstrap.cc:492] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: No bootstrap required, opened a new log
I20260812 14:42:55.203529 31584 ts_tablet_manager.cc:1397] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260812 14:42:55.203543 31636 ts_tablet_manager.cc:1905] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.203667 31584 raft_consensus.cc:348] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.203737 31584 raft_consensus.cc:374] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.203774 31584 raft_consensus.cc:729] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25b1b48ac743639002c30cc6ac684a, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.203810 31584 consensus_queue.cc:260] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.203893 31584 ts_tablet_manager.cc:1428] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:55.204041 31581 tablet_bootstrap.cc:492] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: No bootstrap required, opened a new log
I20260812 14:42:55.204082 31581 ts_tablet_manager.cc:1397] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:42:55.204211 31581 raft_consensus.cc:348] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.204276 31581 raft_consensus.cc:374] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.204293 31581 raft_consensus.cc:729] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.204336 31581 consensus_queue.cc:260] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.204432 31581 ts_tablet_manager.cc:1428] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.205003 31636 ts_tablet_manager.cc:1918] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:55.205075 31636 log.cc:1196] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/27e299ba118b4badbd0731599bb30134
I20260812 14:42:55.205317 31636 ts_tablet_manager.cc:1939] T 27e299ba118b4badbd0731599bb30134 P aca95d5482be426f8b28ba89e76e508e: Deleting consensus metadata
I20260812 14:42:55.205665 30549 catalog_manager.cc:4841] TS aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735): tablet 27e299ba118b4badbd0731599bb30134 (table test-table [id=87a7939de5264d12b31d4a25a15e1559]) successfully deleted
I20260812 14:42:55.205720 31637 ts_tablet_manager.cc:1905] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.206784 31637 ts_tablet_manager.cc:1918] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:55.206858 31637 log.cc:1196] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/27e299ba118b4badbd0731599bb30134
I20260812 14:42:55.207011 31578 tablet_bootstrap.cc:492] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: No bootstrap required, opened a new log
I20260812 14:42:55.207067 31578 ts_tablet_manager.cc:1397] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260812 14:42:55.207093 31637 ts_tablet_manager.cc:1939] T 27e299ba118b4badbd0731599bb30134 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting consensus metadata
I20260812 14:42:55.207198 31578 raft_consensus.cc:348] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.207264 31578 raft_consensus.cc:374] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.207284 31578 raft_consensus.cc:729] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5cb51c9c5f4e95b8c8d19c9c31b2e2, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.207321 31578 consensus_queue.cc:260] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.207401 31578 ts_tablet_manager.cc:1428] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.207563 30540 catalog_manager.cc:4841] TS bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843): tablet 27e299ba118b4badbd0731599bb30134 (table test-table [id=87a7939de5264d12b31d4a25a15e1559]) successfully deleted
W20260812 14:42:55.215309 31266 tablet.cc:2369] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:42:55.323346 30975 tablet.cc:2369] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:42:55.361668 31526 tablet.cc:2369] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:42:55.443034 31592 raft_consensus.cc:482] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:55.443128 31592 raft_consensus.cc:504] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.443420 31592 leader_election.cc:288] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437), aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735)
I20260812 14:42:55.446174 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.443568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36437 (local address 127.15.227.195:55363)
0812 14:42:55.443618 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.443631 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.443680 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.443859 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.443860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.443862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.443935 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.443938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.444566 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.444567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.445394 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.445397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.445925 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.445926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.446002 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.446005 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.446076 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.446107 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 14:42:55.446185 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.443686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:55363 (local address 127.15.227.193:36437)
0812 14:42:55.443747 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.443749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.443758 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:55.443790 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.443791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.443800 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.443855 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.444001 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.444498 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.445472 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.445855 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.445907 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.445916 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.446083 (+ 167us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.446117 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:42:55.446264 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.443760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:36179 (local address 127.15.227.194:44735)
0812 14:42:55.443817 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.443819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.443828 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:55.443854 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.443855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.443866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.443921 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.444096 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.444547 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.445557 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.445922 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.445990 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.445998 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.446170 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.446204 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:42:55.446282 31651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.443596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:44735 (local address 127.15.227.195:36179)
0812 14:42:55.443737 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.443746 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.443796 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.443932 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.443933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.443935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.444030 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.444032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.444625 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.444625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.445477 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.445479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.446033 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.446034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.446098 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.446103 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.446165 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.446201 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":75,"threads_started":1}
I20260812 14:42:55.446461 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3f08b16304648e3ab795aa62b2c3382" candidate_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a" is_pre_election: true
I20260812 14:42:55.446549 30875 raft_consensus.cc:2454] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc5cb51c9c5f4e95b8c8d19c9c31b2e2 in term 0.
I20260812 14:42:55.446609 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3f08b16304648e3ab795aa62b2c3382" candidate_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca95d5482be426f8b28ba89e76e508e" is_pre_election: true
I20260812 14:42:55.446715 31312 leader_election.cc:302] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a25b1b48ac743639002c30cc6ac684a, bc5cb51c9c5f4e95b8c8d19c9c31b2e2; no voters:
I20260812 14:42:55.446753 31162 raft_consensus.cc:2454] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc5cb51c9c5f4e95b8c8d19c9c31b2e2 in term 0.
I20260812 14:42:55.446846 31592 raft_consensus.cc:2781] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:55.446885 31592 raft_consensus.cc:482] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:55.446903 31592 raft_consensus.cc:3037] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.447576 31592 raft_consensus.cc:504] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.447705 31592 leader_election.cc:288] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [CANDIDATE]: Term 1 election: Requested vote from peers 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437), aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735)
I20260812 14:42:55.447834 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3f08b16304648e3ab795aa62b2c3382" candidate_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca95d5482be426f8b28ba89e76e508e"
I20260812 14:42:55.447829 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c3f08b16304648e3ab795aa62b2c3382" candidate_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a"
I20260812 14:42:55.447898 31162 raft_consensus.cc:3037] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.447901 30875 raft_consensus.cc:3037] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.448482 30875 raft_consensus.cc:2454] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc5cb51c9c5f4e95b8c8d19c9c31b2e2 in term 1.
I20260812 14:42:55.448590 31162 raft_consensus.cc:2454] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc5cb51c9c5f4e95b8c8d19c9c31b2e2 in term 1.
I20260812 14:42:55.448647 31312 leader_election.cc:302] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a25b1b48ac743639002c30cc6ac684a, bc5cb51c9c5f4e95b8c8d19c9c31b2e2; no voters:
I20260812 14:42:55.448732 31592 raft_consensus.cc:2781] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:55.448892 31592 raft_consensus.cc:686] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 LEADER]: Becoming Leader. State: Replica: bc5cb51c9c5f4e95b8c8d19c9c31b2e2, State: Running, Role: LEADER
I20260812 14:42:55.448968 31592 consensus_queue.cc:237] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.449622 30564 catalog_manager.cc:5495] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195). New cstate: current_term: 1 leader_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } health_report { overall_health: HEALTHY } } }
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545774, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.254@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:55.500514 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.494552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33953 (local address 127.0.0.1:59090)
0812 14:42:55.494724 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.494733 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.494799 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.494911 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.494912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.495210 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:55.495250 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.495252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.495849 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.495850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.496353 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.496354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.496645 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.496645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.496690 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.497148 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:55.497153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:55.498400 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:55.499594 (+ 1194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:55.499601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:55.499603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:55.499779 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:55.499940 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:55.499941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:55.499941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:55.499992 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:55.500190 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:55.500198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:55.500243 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.500299 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.500405 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260812 14:42:55.500553 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.494607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.15.227.254:33953)
0812 14:42:55.494680 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.494683 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.494784 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:42:55.494843 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.494843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.494853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.494895 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:55.495322 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.495786 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.496418 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.496591 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.496649 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.498480 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:55.498481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:55.498482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:55.498488 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:55.499527 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.499840 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.499841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.499841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.499891 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.500039 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.500040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.500040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.500138 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:55.500195 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.500328 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.500454 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":46}
I20260812 14:42:55.502187 30564 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:59090:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 14:42:55.502365 30564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:55.503651 31403 tablet_service.cc:1467] Processing CreateTablet for tablet ba4e090c29224b6eba3747d9035bf5fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55cc57c7ba5243a5a5e1aac04774aeec]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:55.503664 30839 tablet_service.cc:1467] Processing CreateTablet for tablet ba4e090c29224b6eba3747d9035bf5fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55cc57c7ba5243a5a5e1aac04774aeec]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:55.503672 31121 tablet_service.cc:1467] Processing CreateTablet for tablet ba4e090c29224b6eba3747d9035bf5fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=55cc57c7ba5243a5a5e1aac04774aeec]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 14:42:55.503801 30839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba4e090c29224b6eba3747d9035bf5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.503808 31403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba4e090c29224b6eba3747d9035bf5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.503808 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba4e090c29224b6eba3747d9035bf5fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.504947 31584 tablet_bootstrap.cc:492] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: Bootstrap starting.
I20260812 14:42:55.505175 31581 tablet_bootstrap.cc:492] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: Bootstrap starting.
I20260812 14:42:55.505187 31578 tablet_bootstrap.cc:492] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Bootstrap starting.
I20260812 14:42:55.505371 31584 tablet_bootstrap.cc:654] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.505501 31581 tablet_bootstrap.cc:654] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.505517 31578 tablet_bootstrap.cc:654] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.505990 31584 tablet_bootstrap.cc:492] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: No bootstrap required, opened a new log
I20260812 14:42:55.506065 31584 ts_tablet_manager.cc:1397] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:55.506158 31578 tablet_bootstrap.cc:492] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: No bootstrap required, opened a new log
I20260812 14:42:55.506206 31578 ts_tablet_manager.cc:1397] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:55.506232 31584 raft_consensus.cc:348] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.506306 31584 raft_consensus.cc:374] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.506310 31581 tablet_bootstrap.cc:492] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: No bootstrap required, opened a new log
I20260812 14:42:55.506330 31584 raft_consensus.cc:729] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25b1b48ac743639002c30cc6ac684a, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.506354 31581 ts_tablet_manager.cc:1397] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:55.506352 31578 raft_consensus.cc:348] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.506368 31584 consensus_queue.cc:260] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.506412 31578 raft_consensus.cc:374] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.506430 31578 raft_consensus.cc:729] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5cb51c9c5f4e95b8c8d19c9c31b2e2, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.506454 31584 ts_tablet_manager.cc:1428] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.506466 31578 consensus_queue.cc:260] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.506465 31581 raft_consensus.cc:348] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.506525 31581 raft_consensus.cc:374] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.506543 31581 raft_consensus.cc:729] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.506546 31578 ts_tablet_manager.cc:1428] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.506523 31594 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.506580 31581 consensus_queue.cc:260] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.506618 31594 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:55.506673 31581 ts_tablet_manager.cc:1428] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.506705 31627 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.506789 31627 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:55.506829 31592 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.506904 31592 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:55.578814 31594 raft_consensus.cc:482] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:55.578894 31594 raft_consensus.cc:504] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.579200 31594 leader_election.cc:288] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735), bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843)
I20260812 14:42:55.581967 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.579433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:60239 (local address 127.15.227.195:41843)
0812 14:42:55.579521 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.579523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.579565 (+ 42us) server_negotiation.cc:407] Connection header received
0812 14:42:55.579643 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.579644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.579654 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.579697 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.579847 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.580295 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.581312 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.581667 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.581727 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.581736 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.581870 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.581904 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:42:55.581979 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.579418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:51989 (local address 127.15.227.194:44735)
0812 14:42:55.579458 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.579460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.579564 (+ 104us) server_negotiation.cc:407] Connection header received
0812 14:42:55.579650 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.579652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.579661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.579708 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:42:55.579830 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.580298 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.581333 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.581684 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.581744 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.581752 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.581880 (+ 128us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.581913 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:42:55.582038 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.579386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:41843 (local address 127.15.227.193:60239)
0812 14:42:55.579508 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.579519 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.579581 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.579707 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.579708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.579709 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.579789 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.579791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.580402 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.580403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.581227 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.581229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.581738 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.581739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.581808 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.581812 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.581856 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.581898 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260812 14:42:55.582190 31676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.579363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:44735 (local address 127.15.227.193:51989)
0812 14:42:55.579495 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.579507 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.579589 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.579717 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.579718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.579720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:42:55.579777 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.579779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.580372 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.580373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.581247 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.581250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.581757 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.581758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.581820 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.581823 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.581867 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.581911 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":50,"thread_start_us":79,"threads_started":1}
I20260812 14:42:55.582337 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba4e090c29224b6eba3747d9035bf5fd" candidate_uuid: "5a25b1b48ac743639002c30cc6ac684a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" is_pre_election: true
I20260812 14:42:55.582413 31448 raft_consensus.cc:2454] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a25b1b48ac743639002c30cc6ac684a in term 0.
I20260812 14:42:55.582472 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba4e090c29224b6eba3747d9035bf5fd" candidate_uuid: "5a25b1b48ac743639002c30cc6ac684a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca95d5482be426f8b28ba89e76e508e" is_pre_election: true
I20260812 14:42:55.582561 31162 raft_consensus.cc:2454] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a25b1b48ac743639002c30cc6ac684a in term 0.
I20260812 14:42:55.582568 30740 leader_election.cc:302] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a25b1b48ac743639002c30cc6ac684a, bc5cb51c9c5f4e95b8c8d19c9c31b2e2; no voters:
I20260812 14:42:55.582690 31594 raft_consensus.cc:2781] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:55.582746 31594 raft_consensus.cc:482] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:55.582777 31594 raft_consensus.cc:3037] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.583389 31594 raft_consensus.cc:504] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.583496 31594 leader_election.cc:288] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [CANDIDATE]: Term 1 election: Requested vote from peers aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735), bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843)
I20260812 14:42:55.583659 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba4e090c29224b6eba3747d9035bf5fd" candidate_uuid: "5a25b1b48ac743639002c30cc6ac684a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2"
I20260812 14:42:55.583659 31162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba4e090c29224b6eba3747d9035bf5fd" candidate_uuid: "5a25b1b48ac743639002c30cc6ac684a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aca95d5482be426f8b28ba89e76e508e"
I20260812 14:42:55.583730 31162 raft_consensus.cc:3037] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.583730 31448 raft_consensus.cc:3037] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.584467 31162 raft_consensus.cc:2454] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a25b1b48ac743639002c30cc6ac684a in term 1.
I20260812 14:42:55.584590 31448 raft_consensus.cc:2454] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a25b1b48ac743639002c30cc6ac684a in term 1.
I20260812 14:42:55.584647 30744 leader_election.cc:302] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a25b1b48ac743639002c30cc6ac684a, aca95d5482be426f8b28ba89e76e508e; no voters:
I20260812 14:42:55.584739 31594 raft_consensus.cc:2781] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:55.584849 31594 raft_consensus.cc:686] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [term 1 LEADER]: Becoming Leader. State: Replica: 5a25b1b48ac743639002c30cc6ac684a, State: Running, Role: LEADER
I20260812 14:42:55.584918 31594 consensus_queue.cc:237] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.585237 31685 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: TxnStatusTablet state changed. Reason: New leader 5a25b1b48ac743639002c30cc6ac684a. Latest consensus state: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.585332 31685 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:55.585556 31688 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 14:42:55.585655 31688 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 14:42:55.585603 30564 catalog_manager.cc:5495] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193). New cstate: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:55.599929 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.597684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.15.227.193:36437)
0812 14:42:55.597733 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.597736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.597742 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:42:55.597768 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.597769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.597786 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.597840 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.597963 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.598495 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.599205 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.599402 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.599454 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.599620 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.599719 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.599778 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.599828 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.599848 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:42:55.599938 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.597487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36437 (local address 127.0.0.1:32888)
0812 14:42:55.597550 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.597561 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.597606 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.597860 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.597861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.597862 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:55.597907 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.597909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.598564 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.598565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.599117 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.599119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.599463 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.599464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.599531 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.599545 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.599776 (+ 231us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.599784 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.599826 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.599875 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:42:55.600893 31448 raft_consensus.cc:1261] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Refusing update from remote peer 5a25b1b48ac743639002c30cc6ac684a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.600975 31162 raft_consensus.cc:1261] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Refusing update from remote peer 5a25b1b48ac743639002c30cc6ac684a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.601101 31685 consensus_queue.cc:1035] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.601209 31594 consensus_queue.cc:1035] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a [LEADER]: Connected to new peer: Peer: permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.601802 31592 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: TxnStatusTablet state changed. Reason: New leader 5a25b1b48ac743639002c30cc6ac684a. Latest consensus state: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.601817 31627 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: TxnStatusTablet state changed. Reason: New leader 5a25b1b48ac743639002c30cc6ac684a. Latest consensus state: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.601881 31592 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:55.601895 31627 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:55.602252 31685 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.602325 31685 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:55.602592 31592 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.602679 31592 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:55.603317 31594 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.603400 31594 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P 5a25b1b48ac743639002c30cc6ac684a: This TxnStatusTablet replica's current role is: LEADER
I20260812 14:42:55.603796 31627 tablet_replica.cc:439] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5a25b1b48ac743639002c30cc6ac684a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } } }
I20260812 14:42:55.603873 31627 tablet_replica.cc:442] T ba4e090c29224b6eba3747d9035bf5fd P aca95d5482be426f8b28ba89e76e508e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 14:42:55.606662 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.604201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.15.227.195:41843)
0812 14:42:55.604331 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.604332 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.604339 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:55.604369 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.604369 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.604389 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.604447 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.604551 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.605262 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.605923 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.606232 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.606302 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.606397 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.606502 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.606562 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.606579 (+ 17us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.606596 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260812 14:42:55.611054 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.608798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:43329 (local address 127.15.227.193:36437)
0812 14:42:55.608832 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.608833 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.608844 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:55.608907 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.608908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.608914 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.608953 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.609052 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.609453 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.610458 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.610663 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.610722 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.610876 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.610900 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.610938 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.611000 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.611012 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:42:55.611097 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.608747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36437 (local address 127.15.227.195:43329)
0812 14:42:55.608805 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.608815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.608856 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.608956 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.608957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.608958 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:55.609007 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.609008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.609509 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.609509 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.610383 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.610385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.610730 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.610731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.610793 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.610807 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.610950 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.610951 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.610997 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.611032 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:42:55.613215 31162 raft_consensus.cc:1261] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Refusing update from remote peer bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.613216 30875 raft_consensus.cc:1261] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Refusing update from remote peer bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.613447 31592 consensus_queue.cc:1035] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.613576 31655 consensus_queue.cc:1035] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.614547 31615 mvcc.cc:205] Tried to move back new op lower bound from 7317691496910065664 to 7317691496239439872. Current Snapshot: MvccSnapshot[applied={T|T < 7317691496910065664}]
I20260812 14:42:55.621423 31573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.618931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:57505 (local address 127.15.227.195:41843)
0812 14:42:55.618962 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.618964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.619190 (+ 226us) server_negotiation.cc:407] Connection header received
0812 14:42:55.619251 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.619252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.619260 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.619311 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.619449 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.620003 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.620707 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.620897 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.620947 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.621120 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.621149 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.621198 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.621344 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.621366 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:42:55.621451 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.618888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:41843 (local address 127.15.227.193:57505)
0812 14:42:55.619136 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.619146 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.619200 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.619328 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.619329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.619330 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:55.619395 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.619396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.620072 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.620073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.620633 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.620635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.620968 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.620969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.621032 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.621046 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.621219 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.621222 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.621316 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.621359 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":74,"threads_started":1}
I20260812 14:42:55.634809 30564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.634896 30564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.635468 30564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Sending DeleteTablet for 3 replicas of tablet c3f08b16304648e3ab795aa62b2c3382
I20260812 14:42:55.635704 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet c3f08b16304648e3ab795aa62b2c3382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:40792
I20260812 14:42:55.635805 31636 tablet_replica.cc:330] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: stopping tablet replica
I20260812 14:42:55.635794 31403 tablet_service.cc:1514] Processing DeleteTablet for tablet c3f08b16304648e3ab795aa62b2c3382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:37384
I20260812 14:42:55.635865 31636 raft_consensus.cc:2229] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.635900 31636 raft_consensus.cc:2258] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.635905 31637 tablet_replica.cc:330] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: stopping tablet replica
I20260812 14:42:55.635957 31637 raft_consensus.cc:2229] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:55.636030 31637 raft_consensus.cc:2258] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.636305 31636 ts_tablet_manager.cc:1905] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.636346 31637 ts_tablet_manager.cc:1905] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.637813 31637 ts_tablet_manager.cc:1918] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:55.637832 31636 ts_tablet_manager.cc:1918] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:55.637868 31637 log.cc:1196] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/c3f08b16304648e3ab795aa62b2c3382
I20260812 14:42:55.637882 31636 log.cc:1196] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/c3f08b16304648e3ab795aa62b2c3382
I20260812 14:42:55.638103 31637 ts_tablet_manager.cc:1939] T c3f08b16304648e3ab795aa62b2c3382 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting consensus metadata
I20260812 14:42:55.638157 31636 ts_tablet_manager.cc:1939] T c3f08b16304648e3ab795aa62b2c3382 P aca95d5482be426f8b28ba89e76e508e: Deleting consensus metadata
I20260812 14:42:55.638443 30549 catalog_manager.cc:4841] TS aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735): tablet c3f08b16304648e3ab795aa62b2c3382 (table test-table [id=d49ef647062543ec8cc53b7f9ea2adf3]) successfully deleted
I20260812 14:42:55.638449 30540 catalog_manager.cc:4841] TS bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843): tablet c3f08b16304648e3ab795aa62b2c3382 (table test-table [id=d49ef647062543ec8cc53b7f9ea2adf3]) successfully deleted
I20260812 14:42:55.639187 30839 tablet_service.cc:1514] Processing DeleteTablet for tablet c3f08b16304648e3ab795aa62b2c3382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:32884
I20260812 14:42:55.639289 31641 tablet_replica.cc:330] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: stopping tablet replica
I20260812 14:42:55.639350 31641 raft_consensus.cc:2229] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.639382 31641 raft_consensus.cc:2258] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.640064 31641 ts_tablet_manager.cc:1905] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.641525 31641 ts_tablet_manager.cc:1918] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:55.641587 31641 log.cc:1196] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/c3f08b16304648e3ab795aa62b2c3382
I20260812 14:42:55.641805 31641 ts_tablet_manager.cc:1939] T c3f08b16304648e3ab795aa62b2c3382 P 5a25b1b48ac743639002c30cc6ac684a: Deleting consensus metadata
I20260812 14:42:55.642066 30540 catalog_manager.cc:4841] TS 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437): tablet c3f08b16304648e3ab795aa62b2c3382 (table test-table [id=d49ef647062543ec8cc53b7f9ea2adf3]) successfully deleted
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:42:55.666234 30669 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:55.659682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59104 (local address 127.15.227.254:33953)
0812 14:42:55.659729 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.659731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.659877 (+ 146us) server_negotiation.cc:407] Connection header received
0812 14:42:55.659933 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.659934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.659947 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.660010 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:55.660682 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.661453 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.662558 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.662865 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.662929 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.666171 (+ 3242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59104: BlockingRecv error: recv got EOF from 127.0.0.1:59104 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[30484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM
I20260812 14:42:55.678388 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.670232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.15.227.254:33953)
0812 14:42:55.670276 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.670278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.670320 (+ 42us) server_negotiation.cc:407] Connection header received
0812 14:42:55.670382 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.670383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.670392 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.670450 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:55.670990 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.671684 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.672689 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.672966 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.673045 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.675616 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:55.675618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:55.675618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:55.675625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:55.677194 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.677580 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.677581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.677581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.677637 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:55.677839 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:55.677840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:55.677840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:55.677962 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:55.678074 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.678225 (+ 151us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.678304 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:42:55.679853 30564 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:55.680023 30564 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:55.681427 30839 tablet_service.cc:1467] Processing CreateTablet for tablet 8d230dfce1214e67a230445605de8dc6 (DEFAULT_TABLE table=test-table [id=d3dc273ac18b4249bc9cb57e7b41ab86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.681511 31403 tablet_service.cc:1467] Processing CreateTablet for tablet 8d230dfce1214e67a230445605de8dc6 (DEFAULT_TABLE table=test-table [id=d3dc273ac18b4249bc9cb57e7b41ab86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.681568 30839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d230dfce1214e67a230445605de8dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.681629 31403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d230dfce1214e67a230445605de8dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.682286 31121 tablet_service.cc:1467] Processing CreateTablet for tablet 8d230dfce1214e67a230445605de8dc6 (DEFAULT_TABLE table=test-table [id=d3dc273ac18b4249bc9cb57e7b41ab86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.682412 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d230dfce1214e67a230445605de8dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.683192 31578 tablet_bootstrap.cc:492] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Bootstrap starting.
I20260812 14:42:55.683576 31578 tablet_bootstrap.cc:654] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.683732 31581 tablet_bootstrap.cc:492] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: Bootstrap starting.
I20260812 14:42:55.684049 31581 tablet_bootstrap.cc:654] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.685456 31584 tablet_bootstrap.cc:492] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: Bootstrap starting.
I20260812 14:42:55.685755 31581 tablet_bootstrap.cc:492] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: No bootstrap required, opened a new log
I20260812 14:42:55.685807 31581 ts_tablet_manager.cc:1397] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:42:55.685864 31584 tablet_bootstrap.cc:654] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.685949 31581 raft_consensus.cc:348] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.686131 31581 raft_consensus.cc:374] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.686165 31581 raft_consensus.cc:729] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.686208 31581 consensus_queue.cc:260] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.686304 31581 ts_tablet_manager.cc:1428] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.686367 31627 raft_consensus.cc:482] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:55.686395 31627 raft_consensus.cc:504] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.686491 31627 leader_election.cc:288] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437), bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843)
I20260812 14:42:55.686721 31584 tablet_bootstrap.cc:492] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: No bootstrap required, opened a new log
I20260812 14:42:55.686762 31584 ts_tablet_manager.cc:1397] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:55.686838 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d230dfce1214e67a230445605de8dc6" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" is_pre_election: true
I20260812 14:42:55.686893 31584 raft_consensus.cc:348] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.686955 31584 raft_consensus.cc:374] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.686973 31584 raft_consensus.cc:729] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25b1b48ac743639002c30cc6ac684a, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.687014 31584 consensus_queue.cc:260] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.687089 31584 ts_tablet_manager.cc:1428] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.687345 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d230dfce1214e67a230445605de8dc6" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a" is_pre_election: true
I20260812 14:42:55.687422 30875 raft_consensus.cc:2454] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 0.
W20260812 14:42:55.687572 31019 leader_election.cc:341] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843): Illegal state: must be running to vote when last-logged opid is not known
I20260812 14:42:55.687686 31019 leader_election.cc:302] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a25b1b48ac743639002c30cc6ac684a, aca95d5482be426f8b28ba89e76e508e; no voters: bc5cb51c9c5f4e95b8c8d19c9c31b2e2
I20260812 14:42:55.687757 31593 raft_consensus.cc:2781] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:55.687800 31593 raft_consensus.cc:482] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:55.687822 31593 raft_consensus.cc:3037] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.688490 31593 raft_consensus.cc:504] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.688599 31593 leader_election.cc:288] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 election: Requested vote from peers 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437), bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843)
I20260812 14:42:55.689043 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d230dfce1214e67a230445605de8dc6" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2"
W20260812 14:42:55.689253 31019 leader_election.cc:341] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843): Illegal state: must be running to vote when last-logged opid is not known
I20260812 14:42:55.689263 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d230dfce1214e67a230445605de8dc6" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a"
I20260812 14:42:55.689327 30875 raft_consensus.cc:3037] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.689982 30875 raft_consensus.cc:2454] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 1.
I20260812 14:42:55.690167 31019 leader_election.cc:302] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a25b1b48ac743639002c30cc6ac684a, aca95d5482be426f8b28ba89e76e508e; no voters: bc5cb51c9c5f4e95b8c8d19c9c31b2e2
I20260812 14:42:55.690306 31593 raft_consensus.cc:2781] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:55.690366 31593 raft_consensus.cc:686] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 1 LEADER]: Becoming Leader. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Running, Role: LEADER
I20260812 14:42:55.690398 31593 consensus_queue.cc:237] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.690830 30564 catalog_manager.cc:5495] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e reported cstate change: term changed from 0 to 1, leader changed from <none> to aca95d5482be426f8b28ba89e76e508e (127.15.227.194). New cstate: current_term: 1 leader_uuid: "aca95d5482be426f8b28ba89e76e508e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:55.694389 31578 tablet_bootstrap.cc:492] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: No bootstrap required, opened a new log
I20260812 14:42:55.694439 31578 ts_tablet_manager.cc:1397] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20260812 14:42:55.694564 31578 raft_consensus.cc:348] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.694619 31578 raft_consensus.cc:374] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.694638 31578 raft_consensus.cc:729] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5cb51c9c5f4e95b8c8d19c9c31b2e2, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.694675 31578 consensus_queue.cc:260] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.694752 31578 ts_tablet_manager.cc:1428] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.695905 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.692886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.15.227.194:44735)
0812 14:42:55.692945 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.692947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.692954 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:55.692976 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.692977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.692986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.693040 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.693199 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.693924 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.695085 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.695392 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.695457 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.695652 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.695688 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.695742 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.695803 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.695824 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:42:55.696485 31448 raft_consensus.cc:3037] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.696543 30875 raft_consensus.cc:1261] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Refusing update from remote peer aca95d5482be426f8b28ba89e76e508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.696718 31627 consensus_queue.cc:1035] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.697189 31448 raft_consensus.cc:1261] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Refusing update from remote peer aca95d5482be426f8b28ba89e76e508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.697399 31627 consensus_queue.cc:1035] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.698365 31610 mvcc.cc:205] Tried to move back new op lower bound from 7317691497251733504 to 7317691497228161024. Current Snapshot: MvccSnapshot[applied={T|T < 7317691497251733504}]
I20260812 14:42:55.699749 30564 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.699833 30564 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.700524 30564 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Sending DeleteTablet for 3 replicas of tablet 8d230dfce1214e67a230445605de8dc6
I20260812 14:42:55.700776 30839 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d230dfce1214e67a230445605de8dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:32884
I20260812 14:42:55.700851 31401 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d230dfce1214e67a230445605de8dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:37384
I20260812 14:42:55.700906 31641 tablet_replica.cc:330] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: stopping tablet replica
I20260812 14:42:55.700948 31641 raft_consensus.cc:2229] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.700984 31641 raft_consensus.cc:2258] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.701103 31637 tablet_replica.cc:330] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: stopping tablet replica
I20260812 14:42:55.701151 31637 raft_consensus.cc:2229] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.701184 31637 raft_consensus.cc:2258] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.701187 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d230dfce1214e67a230445605de8dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:40792
I20260812 14:42:55.701263 31636 tablet_replica.cc:330] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: stopping tablet replica
I20260812 14:42:55.700856 31403 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d230dfce1214e67a230445605de8dc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:37384
I20260812 14:42:55.701304 31636 raft_consensus.cc:2229] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:55.701352 31636 raft_consensus.cc:2258] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.701471 30566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:42:55.701599 30566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:42:55.701915 31636 ts_tablet_manager.cc:1905] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.703080 31121 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 (DEFAULT_TABLE table=test-table [id=7b377ed7e881456096d1ef2f92a324a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.703186 31636 ts_tablet_manager.cc:1918] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:55.703253 31636 log.cc:1196] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/8d230dfce1214e67a230445605de8dc6
I20260812 14:42:55.703439 31636 ts_tablet_manager.cc:1939] T 8d230dfce1214e67a230445605de8dc6 P aca95d5482be426f8b28ba89e76e508e: Deleting consensus metadata
I20260812 14:42:55.703667 31403 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 (DEFAULT_TABLE table=test-table [id=7b377ed7e881456096d1ef2f92a324a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.703706 30549 catalog_manager.cc:4841] TS aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735): tablet 8d230dfce1214e67a230445605de8dc6 (table test-table [id=d3dc273ac18b4249bc9cb57e7b41ab86]) successfully deleted
I20260812 14:42:55.703758 31403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.704854 31578 tablet_bootstrap.cc:492] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Bootstrap starting.
I20260812 14:42:55.705207 31578 tablet_bootstrap.cc:654] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.705504 30839 tablet_service.cc:1467] Processing CreateTablet for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 (DEFAULT_TABLE table=test-table [id=7b377ed7e881456096d1ef2f92a324a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:42:55.705601 30839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.705808 31578 tablet_bootstrap.cc:492] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: No bootstrap required, opened a new log
I20260812 14:42:55.705848 31578 ts_tablet_manager.cc:1397] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:55.705967 31578 raft_consensus.cc:348] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.706048 31578 raft_consensus.cc:374] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.706065 31578 raft_consensus.cc:729] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc5cb51c9c5f4e95b8c8d19c9c31b2e2, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.706115 31578 consensus_queue.cc:260] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.706204 31578 ts_tablet_manager.cc:1428] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:55.703193 31121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:55.707322 31581 tablet_bootstrap.cc:492] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: Bootstrap starting.
I20260812 14:42:55.707635 31581 tablet_bootstrap.cc:654] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.708575 31584 tablet_bootstrap.cc:492] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a: Bootstrap starting.
I20260812 14:42:55.708953 31584 tablet_bootstrap.cc:654] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:55.709538 31641 ts_tablet_manager.cc:1905] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.710821 31641 ts_tablet_manager.cc:1918] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:55.710871 31641 log.cc:1196] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/8d230dfce1214e67a230445605de8dc6
I20260812 14:42:55.711053 31641 ts_tablet_manager.cc:1939] T 8d230dfce1214e67a230445605de8dc6 P 5a25b1b48ac743639002c30cc6ac684a: Deleting consensus metadata
I20260812 14:42:55.711395 30540 catalog_manager.cc:4841] TS 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437): tablet 8d230dfce1214e67a230445605de8dc6 (table test-table [id=d3dc273ac18b4249bc9cb57e7b41ab86]) successfully deleted
I20260812 14:42:55.711619 31637 ts_tablet_manager.cc:1905] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.712920 31637 ts_tablet_manager.cc:1918] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:42:55.712970 31637 log.cc:1196] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/8d230dfce1214e67a230445605de8dc6
I20260812 14:42:55.713174 31637 ts_tablet_manager.cc:1939] T 8d230dfce1214e67a230445605de8dc6 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting consensus metadata
I20260812 14:42:55.713511 30540 catalog_manager.cc:4841] TS bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843): tablet 8d230dfce1214e67a230445605de8dc6 (table test-table [id=d3dc273ac18b4249bc9cb57e7b41ab86]) successfully deleted
W20260812 14:42:55.713655 30540 catalog_manager.cc:4808] TS bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843): delete failed for tablet 8d230dfce1214e67a230445605de8dc6 because the tablet was not found. No further retry: Not found: Tablet not found: 8d230dfce1214e67a230445605de8dc6
I20260812 14:42:55.714121 31584 tablet_bootstrap.cc:492] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a: No bootstrap required, opened a new log
I20260812 14:42:55.714184 31584 ts_tablet_manager.cc:1397] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260812 14:42:55.714308 31581 tablet_bootstrap.cc:492] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: No bootstrap required, opened a new log
I20260812 14:42:55.714354 31581 ts_tablet_manager.cc:1397] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260812 14:42:55.714322 31584 raft_consensus.cc:348] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.714385 31584 raft_consensus.cc:374] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.714404 31584 raft_consensus.cc:729] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a25b1b48ac743639002c30cc6ac684a, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.714444 31584 consensus_queue.cc:260] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.714445 31581 raft_consensus.cc:348] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.714504 31581 raft_consensus.cc:374] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:55.714524 31581 raft_consensus.cc:729] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Initialized, Role: FOLLOWER
I20260812 14:42:55.714529 31584 ts_tablet_manager.cc:1428] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.714562 31581 consensus_queue.cc:260] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.714651 31581 ts_tablet_manager.cc:1428] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:55.765695 31593 raft_consensus.cc:482] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:42:55.765789 31593 raft_consensus.cc:504] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.765954 31593 leader_election.cc:288] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437), bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843)
I20260812 14:42:55.766165 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8f55c8d3c3494b914aaa5b6df0fd18" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" is_pre_election: true
I20260812 14:42:55.766160 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8f55c8d3c3494b914aaa5b6df0fd18" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a" is_pre_election: true
I20260812 14:42:55.766270 30875 raft_consensus.cc:2454] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 0.
I20260812 14:42:55.766270 31448 raft_consensus.cc:2454] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 0.
I20260812 14:42:55.766413 31019 leader_election.cc:302] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aca95d5482be426f8b28ba89e76e508e, bc5cb51c9c5f4e95b8c8d19c9c31b2e2; no voters:
I20260812 14:42:55.766527 31593 raft_consensus.cc:2781] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:42:55.766568 31593 raft_consensus.cc:482] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:42:55.766592 31593 raft_consensus.cc:3037] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.767257 31593 raft_consensus.cc:504] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.767376 31593 leader_election.cc:288] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 election: Requested vote from peers 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437), bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843)
I20260812 14:42:55.767504 30875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8f55c8d3c3494b914aaa5b6df0fd18" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a25b1b48ac743639002c30cc6ac684a"
I20260812 14:42:55.767524 31448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e8f55c8d3c3494b914aaa5b6df0fd18" candidate_uuid: "aca95d5482be426f8b28ba89e76e508e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2"
I20260812 14:42:55.767576 30875 raft_consensus.cc:3037] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.767589 31448 raft_consensus.cc:3037] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:55.768143 31448 raft_consensus.cc:2454] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 1.
I20260812 14:42:55.768201 30875 raft_consensus.cc:2454] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aca95d5482be426f8b28ba89e76e508e in term 1.
I20260812 14:42:55.768324 31019 leader_election.cc:302] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aca95d5482be426f8b28ba89e76e508e, bc5cb51c9c5f4e95b8c8d19c9c31b2e2; no voters:
I20260812 14:42:55.768431 31593 raft_consensus.cc:2781] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:55.768498 31593 raft_consensus.cc:686] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 1 LEADER]: Becoming Leader. State: Replica: aca95d5482be426f8b28ba89e76e508e, State: Running, Role: LEADER
I20260812 14:42:55.768550 31593 consensus_queue.cc:237] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } }
I20260812 14:42:55.769085 30566 catalog_manager.cc:5495] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e reported cstate change: term changed from 0 to 1, leader changed from <none> to aca95d5482be426f8b28ba89e76e508e (127.15.227.194). New cstate: current_term: 1 leader_uuid: "aca95d5482be426f8b28ba89e76e508e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aca95d5482be426f8b28ba89e76e508e" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 44735 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 } health_report { overall_health: UNKNOWN } } }
I20260812 14:42:55.782502 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.780429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:55823 (local address 127.15.227.193:36437)
0812 14:42:55.780485 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.780487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.780611 (+ 124us) server_negotiation.cc:407] Connection header received
0812 14:42:55.780660 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.780660 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.780669 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.780710 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.780788 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.781264 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.781964 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.782171 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.782215 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.782345 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.782368 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.782399 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.782440 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.782456 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:42:55.782529 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.780374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36437 (local address 127.15.227.194:55823)
0812 14:42:55.780550 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.780559 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.780611 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.780713 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.780713 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.780714 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:55.780755 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.780756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.781320 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.781321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.781908 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.781910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.782228 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.782229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.782272 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.782286 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.782407 (+ 121us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.782408 (+ 1us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.782436 (+ 28us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.782474 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260812 14:42:55.784232 31448 raft_consensus.cc:1261] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Refusing update from remote peer aca95d5482be426f8b28ba89e76e508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.784267 30875 raft_consensus.cc:1261] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Refusing update from remote peer aca95d5482be426f8b28ba89e76e508e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:42:55.784410 31593 consensus_queue.cc:1035] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc5cb51c9c5f4e95b8c8d19c9c31b2e2" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.784518 31627 consensus_queue.cc:1035] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a25b1b48ac743639002c30cc6ac684a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:42:55.790421 31574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.788371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46241 (local address 127.15.227.194:44735)
0812 14:42:55.788406 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:55.788408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:55.788415 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:42:55.788465 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:55.788466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:55.788475 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:55.788524 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:42:55.788637 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.789121 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.789783 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.789980 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.790085 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.790204 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.790236 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.790290 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:55.790346 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:42:55.790362 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:42:55.790459 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.788335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:44735 (local address 127.15.227.193:46241)
0812 14:42:55.788374 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.788385 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:55.788421 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:55.788538 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:55.788540 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:55.788541 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 14:42:55.788579 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.788580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.789186 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.789187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.789715 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:55.789717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:55.790085 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:55.790086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:55.790143 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:55.790149 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 14:42:55.790293 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 14:42:55.790296 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:42:55.790345 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:42:55.790396 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 14:42:55.796267 30563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.796339 30563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:42:55.796866 30563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3dde5f6d85c4be09869d19092f73630: Sending DeleteTablet for 3 replicas of tablet 1e8f55c8d3c3494b914aaa5b6df0fd18
I20260812 14:42:55.797070 31121 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:40792
I20260812 14:42:55.797101 31403 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:37384
I20260812 14:42:55.797122 30839 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:42:55 UTC) from {username='oryx', principal='oryx/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:32884
I20260812 14:42:55.797168 31636 tablet_replica.cc:330] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: stopping tablet replica
I20260812 14:42:55.797205 31641 tablet_replica.cc:330] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a: stopping tablet replica
I20260812 14:42:55.797254 31636 raft_consensus.cc:2229] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:42:55.797256 31641 raft_consensus.cc:2229] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.797297 31641 raft_consensus.cc:2258] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.797312 31636 raft_consensus.cc:2258] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.797434 31637 tablet_replica.cc:330] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: stopping tablet replica
I20260812 14:42:55.797484 31637 raft_consensus.cc:2229] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:42:55.797528 31637 raft_consensus.cc:2258] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:42:55.797616 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 30711
I20260812 14:42:55.797681 31641 ts_tablet_manager.cc:1905] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P 5a25b1b48ac743639002c30cc6ac684a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.797915 31637 ts_tablet_manager.cc:1905] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.800870 31636 ts_tablet_manager.cc:1905] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:42:55.801856 31637 ts_tablet_manager.cc:1918] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:55.801893 31637 log.cc:1196] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/1e8f55c8d3c3494b914aaa5b6df0fd18
I20260812 14:42:55.802090 31637 ts_tablet_manager.cc:1939] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P bc5cb51c9c5f4e95b8c8d19c9c31b2e2: Deleting consensus metadata
I20260812 14:42:55.802250 31636 ts_tablet_manager.cc:1918] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 14:42:55.802296 31636 log.cc:1196] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/1e8f55c8d3c3494b914aaa5b6df0fd18
I20260812 14:42:55.802367 30540 catalog_manager.cc:4841] TS bc5cb51c9c5f4e95b8c8d19c9c31b2e2 (127.15.227.195:41843): tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 (table test-table [id=7b377ed7e881456096d1ef2f92a324a2]) successfully deleted
I20260812 14:42:55.802476 31636 ts_tablet_manager.cc:1939] T 1e8f55c8d3c3494b914aaa5b6df0fd18 P aca95d5482be426f8b28ba89e76e508e: Deleting consensus metadata
I20260812 14:42:55.802781 30549 catalog_manager.cc:4841] TS aca95d5482be426f8b28ba89e76e508e (127.15.227.194:44735): tablet 1e8f55c8d3c3494b914aaa5b6df0fd18 (table test-table [id=7b377ed7e881456096d1ef2f92a324a2]) successfully deleted
W20260812 14:42:55.808374 30540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:36437 (error 108)
I20260812 14:42:55.808552 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 30995
I20260812 14:42:55.808861 31785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:55.808717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36437 (local address 127.0.0.1:32890)
0812 14:42:55.808796 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:55.808830 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:36437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65,"thread_start_us":37,"threads_started":1}
W20260812 14:42:55.808949 30540 catalog_manager.cc:4584] TS 5a25b1b48ac743639002c30cc6ac684a (127.15.227.193:36437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e8f55c8d3c3494b914aaa5b6df0fd18: Network error: Client connection negotiation failed: client connection to 127.15.227.193:36437: connect: Connection refused (error 111)
I20260812 14:42:55.824770 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 31287
I20260812 14:42:55.836503 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 30517
2026-08-12T14:42:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2621 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[31791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[31791](info): set up 2 sockets
Aug 12 14:42:55 dist-test-slave-q9h9 krb5kdc[31791](info): commencing operation
krb5kdc: starting...
W20260812 14:42:57.027020 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:57 dist-test-slave-q9h9 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:57.136441 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33497
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:57.216122 31822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:57.216342 31822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:57.216364 31822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:57.217468 31822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:57.217502 31822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:57.217515 31822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:57.217526 31822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:57.217542 31822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:57.218976 31822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:57.219177 31822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:57.219352 31822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:42:57.224090 31822 server_base.cc:1034] running on GCE node
W20260812 14:42:57.224064 31839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:57.224179 31834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:57.224066 31836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:57.224479 31822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:57.224783 31822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:57.225957 31822 hybrid_clock.cc:648] HybridClock initialized: now 1786545777225932 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T14:42:57Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1389 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:42:57 dist-test-slave-q9h9 krb5kdc[31848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:42:57 dist-test-slave-q9h9 krb5kdc[31848](info): set up 2 sockets
Aug 12 14:42:57 dist-test-slave-q9h9 krb5kdc[31848](info): commencing operation
krb5kdc: starting...
W20260812 14:42:58.406767 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:42:58 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:42:58.517882 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37937
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:58.596122 31875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:58.596303 31875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:58.596324 31875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:58.597380 31875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:58.597412 31875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:58.597425 31875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:58.597436 31875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:58.597450 31875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:58.598852 31875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:58.599030 31875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:58.599192 31875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:58.603508 31889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:58.603694 31887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:58.603574 31894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:58.603688 31875 server_base.cc:1034] running on GCE node
I20260812 14:42:58.603988 31875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:58.604236 31875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:58.605376 31875 hybrid_clock.cc:648] HybridClock initialized: now 1786545778605357 us; error 33 us; skew 500 ppm
Aug 12 14:42:58 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:58.610772 31875 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:58.611130 31875 webserver.cc:460] Webserver started at http://127.15.227.254:43449/ using document root <none> and password file <none>
I20260812 14:42:58.611331 31875 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:58.611374 31875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:58.611459 31875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:58.612349 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "f54dee5234644ce5affe676d755cab93"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "1f4ee0590035583b30adb4f0f167d137"
server_key_iv: "b14e72857be2b0dd2fdb44aa677f0f03"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.612614 31875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "f54dee5234644ce5affe676d755cab93"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "1f4ee0590035583b30adb4f0f167d137"
server_key_iv: "b14e72857be2b0dd2fdb44aa677f0f03"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.613714 31875 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:42:58.614373 31905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.614521 31875 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.614598 31875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "f54dee5234644ce5affe676d755cab93"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "1f4ee0590035583b30adb4f0f167d137"
server_key_iv: "b14e72857be2b0dd2fdb44aa677f0f03"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.614648 31875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:58.625339 31875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:58.626936 31875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:58.627051 31875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:58.630662 31875 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:37937
I20260812 14:42:58.630705 32018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:37937 every 8 connection(s)
I20260812 14:42:58.630978 31875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:58.631577 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 31875
I20260812 14:42:58.631691 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:42:58.631873 32019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:42:58.631888 16271 external_mini_cluster.cc:1428] Setting key 3564ca732a1f72111a879edadb4dfb1d
I20260812 14:42:58.634771 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Bootstrap starting.
I20260812 14:42:58.635433 32019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Neither blocks nor log segments found. Creating new log.
I20260812 14:42:58.635699 32019 log.cc:824] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:58.636341 32019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: No bootstrap required, opened a new log
I20260812 14:42:58.638379 32019 raft_consensus.cc:348] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
Aug 12 14:42:58 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:58.638531 32019 raft_consensus.cc:374] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:42:58.638564 32019 raft_consensus.cc:729] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54dee5234644ce5affe676d755cab93, State: Initialized, Role: FOLLOWER
I20260812 14:42:58.638680 32019 consensus_queue.cc:260] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
I20260812 14:42:58.638741 32019 raft_consensus.cc:388] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:58.638764 32019 raft_consensus.cc:482] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:58.638809 32019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:42:58.639415 32019 raft_consensus.cc:504] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
I20260812 14:42:58.639519 32019 leader_election.cc:302] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f54dee5234644ce5affe676d755cab93; no voters:
I20260812 14:42:58.639675 32019 leader_election.cc:288] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:42:58.639712 32029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:42:58.639838 32029 raft_consensus.cc:686] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 LEADER]: Becoming Leader. State: Replica: f54dee5234644ce5affe676d755cab93, State: Running, Role: LEADER
I20260812 14:42:58.639919 32019 sys_catalog.cc:564] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:58.639920 32029 consensus_queue.cc:237] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
I20260812 14:42:58.640260 32029 sys_catalog.cc:455] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f54dee5234644ce5affe676d755cab93. Latest consensus state: current_term: 1 leader_uuid: "f54dee5234644ce5affe676d755cab93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } } }
I20260812 14:42:58.640336 32029 sys_catalog.cc:458] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:58.640565 32044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:58.641291 32044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:58.642156 32034 sys_catalog.cc:455] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f54dee5234644ce5affe676d755cab93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } } }
I20260812 14:42:58.642230 32034 sys_catalog.cc:458] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:58.642454 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:58.632338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.15.227.254:37937)
0812 14:42:58.632546 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:58.632554 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:58.632582 (+ 28us) server_negotiation.cc:407] Connection header received
0812 14:42:58.633187 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:58.633196 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:58.633255 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:58.633386 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:58.633964 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.635381 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.636128 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.636687 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.636805 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:58.639050 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:58.639054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:58.639054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:58.639066 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:58.640877 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:58.641262 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:58.641265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:58.641265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:58.641369 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:58.641586 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:58.641588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:58.641588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:58.641873 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:58.642010 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:58.642163 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 14:42:58.642293 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":66,"threads_started":1}
I20260812 14:42:58.643173 32044 catalog_manager.cc:1347] Generated new cluster ID: 2e2837cb38f74d73b68ee6bfbcdf0b84
I20260812 14:42:58.643221 32044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:58.679988 32044 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:42:58.680514 32044 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:58.688887 32044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Generated new TSK 0
I20260812 14:42:58.689041 32044 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:42:58.722074 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37937
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:58.810317 32080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:58.810492 32080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:58.810523 32080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:58.811643 32080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:58.811689 32080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:58.811723 32080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:58.813110 32080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:58.813330 32080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:58.813526 32080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:58.818228 32097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:58.818279 32093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:58.818364 32094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:58.818413 32080 server_base.cc:1034] running on GCE node
I20260812 14:42:58.818629 32080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:58.818842 32080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:58.819973 32080 hybrid_clock.cc:648] HybridClock initialized: now 1786545778819957 us; error 34 us; skew 500 ppm
Aug 12 14:42:58 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:58.822263 32080 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:42:58.822613 32080 webserver.cc:460] Webserver started at http://127.15.227.193:35581/ using document root <none> and password file <none>
I20260812 14:42:58.822795 32080 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:58.822832 32080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:58.822919 32080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:58.823804 32080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "31e3eedd177b4c3bb3df464d68ecccf8"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "e32456aba3649074b63b2a27820ac1a6"
server_key_iv: "aedf5b45c4d0ebbeb1afb4c32bb124ed"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.824069 32080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "31e3eedd177b4c3bb3df464d68ecccf8"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "e32456aba3649074b63b2a27820ac1a6"
server_key_iv: "aedf5b45c4d0ebbeb1afb4c32bb124ed"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.825173 32080 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:42:58.825780 32109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.825918 32080 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:58.825995 32080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "31e3eedd177b4c3bb3df464d68ecccf8"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "e32456aba3649074b63b2a27820ac1a6"
server_key_iv: "aedf5b45c4d0ebbeb1afb4c32bb124ed"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.826079 32080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:58.847108 32080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:58.848624 32080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:58.848747 32080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:58.848929 32080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:58.849191 32080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:58.849225 32080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.849258 32080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:58.849280 32080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.854507 32080 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:38675
I20260812 14:42:58.854574 32314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:38675 every 8 connection(s)
I20260812 14:42:58.854833 32080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:58.856925 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 32080
I20260812 14:42:58.857034 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:42:58.857237 16271 external_mini_cluster.cc:1428] Setting key c90e7c81894eba5e9c11000da820eb8c
Aug 12 14:42:58 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:58.863987 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:58.855487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59417 (local address 127.15.227.254:37937)
0812 14:42:58.855556 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:58.855558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:58.855689 (+ 131us) server_negotiation.cc:407] Connection header received
0812 14:42:58.856154 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:58.856155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:58.856164 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:58.856208 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:58.857208 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.857885 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.859188 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.859386 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.859433 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:58.861549 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:58.861551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:58.861551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:58.861558 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:58.862795 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:58.863143 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:58.863144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:58.863145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:58.863196 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:58.863409 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:58.863410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:58.863410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:58.863513 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:58.863569 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:58.863820 (+ 251us) server_negotiation.cc:299] Negotiation successful
0812 14:42:58.863897 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:42:58.864126 32322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:58.855437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:59417)
0812 14:42:58.855588 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:58.855622 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:58.856055 (+ 433us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:58.856264 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:58.856267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:58.856840 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:58.857122 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:58.857129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.857986 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.857988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:58.859103 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:58.859106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.859456 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.859457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:58.859589 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:58.860041 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:58.860049 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:58.861467 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:58.862865 (+ 1398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:58.862872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:58.862874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:58.863079 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:58.863256 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:58.863257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:58.863257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:58.863355 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:58.863569 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:58.863575 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:58.863701 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 14:42:58.863787 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 14:42:58.863927 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260812 14:42:58.864571 32318 heartbeater.cc:347] Connected to a master server at 127.15.227.254:37937
I20260812 14:42:58.864676 32318 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:58.864859 32318 heartbeater.cc:511] Master 127.15.227.254:37937 requested a full tablet report, sending...
I20260812 14:42:58.865446 31944 ts_manager.cc:194] Registered new tserver with Master: 31e3eedd177b4c3bb3df464d68ecccf8 (127.15.227.193:38675)
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
I20260812 14:42:58.866173 31944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:59417
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:42:58.880311 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37937
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:58.959986 32326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:58.960182 32326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:58.960206 32326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:58.961340 32326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:58.961380 32326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:58.961407 32326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:58.962831 32326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:58.963055 32326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:58.963256 32326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:58.967922 32339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:58.967933 32340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:58.968097 32326 server_base.cc:1034] running on GCE node
W20260812 14:42:58.967916 32346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:58.968343 32326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:58.968561 32326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:58.969730 32326 hybrid_clock.cc:648] HybridClock initialized: now 1786545778969713 us; error 26 us; skew 500 ppm
Aug 12 14:42:58 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:58.972098 32326 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:42:58.972519 32326 webserver.cc:460] Webserver started at http://127.15.227.194:45967/ using document root <none> and password file <none>
I20260812 14:42:58.972723 32326 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:58.972770 32326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:58.972889 32326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:58.973840 32326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "f2c16f82b1ee4b85ba3b7602db92e1ef"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "5c304242eea9143ff0d8e667e32095a7"
server_key_iv: "52b8eeb6246e366c5234ed0e0a2d6362"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.974198 32326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "f2c16f82b1ee4b85ba3b7602db92e1ef"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "5c304242eea9143ff0d8e667e32095a7"
server_key_iv: "52b8eeb6246e366c5234ed0e0a2d6362"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.975401 32326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:58.976035 32354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.976181 32326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:58.976262 32326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "f2c16f82b1ee4b85ba3b7602db92e1ef"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "5c304242eea9143ff0d8e667e32095a7"
server_key_iv: "52b8eeb6246e366c5234ed0e0a2d6362"
server_key_version: "encryptionkey@0"
I20260812 14:42:58.976322 32326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:58.987756 32326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:58.989260 32326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:58.989382 32326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:58.989578 32326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:58.989847 32326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:58.989880 32326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.989905 32326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:58.989931 32326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:58.995379 32326 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:37083
I20260812 14:42:58.995414 32564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:37083 every 8 connection(s)
I20260812 14:42:58.995678 32326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:42:58 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:59.003716 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:58.996332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:60363 (local address 127.15.227.254:37937)
0812 14:42:58.996382 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:58.996384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:58.996568 (+ 184us) server_negotiation.cc:407] Connection header received
0812 14:42:58.996973 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:58.996975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:58.996985 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:58.997034 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:58.997845 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.998335 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.999206 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.999381 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.999422 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.001311 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:59.001313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:59.001313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:59.001322 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:59.002530 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:59.002877 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:59.002879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:59.002879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:59.002931 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:59.003140 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:59.003141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:59.003142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:59.003248 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:59.003304 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:59.003546 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 14:42:59.003629 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:42:59.003847 32569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:58.996315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:60363)
0812 14:42:58.996465 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:58.996496 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:58.996903 (+ 407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:58.997094 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:58.997097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:58.997610 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:58.997782 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:58.997786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.998401 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.998402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:58.999146 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:58.999147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:58.999433 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:58.999433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:58.999528 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:58.999910 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:58.999918 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.001214 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:59.002603 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:59.002610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:59.002612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:59.002813 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:59.002984 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:59.002985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:59.002985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:59.003086 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:59.003301 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:59.003308 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:59.003435 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 14:42:59.003516 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 14:42:59.003658 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260812 14:42:59.004249 32565 heartbeater.cc:347] Connected to a master server at 127.15.227.254:37937
I20260812 14:42:59.004333 32565 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:59.004491 32565 heartbeater.cc:511] Master 127.15.227.254:37937 requested a full tablet report, sending...
I20260812 14:42:59.004684 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 32326
I20260812 14:42:59.004817 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:42:59.004874 31944 ts_manager.cc:194] Registered new tserver with Master: f2c16f82b1ee4b85ba3b7602db92e1ef (127.15.227.194:37083)
I20260812 14:42:59.004999 16271 external_mini_cluster.cc:1428] Setting key 761a6868c4833e15daf2cc4dc90abf8d
I20260812 14:42:59.005342 31944 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:60363
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:42:59.029711 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37937
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:59.109670 32576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:59.109863 32576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:59.109895 32576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:59.111052 32576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:59.111093 32576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:59.111135 32576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:59.112534 32576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:59.112753 32576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:59.112957 32576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:59.117658 32595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.117650 32585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.117725 32589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:59.117827 32576 server_base.cc:1034] running on GCE node
I20260812 14:42:59.118072 32576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:59.118330 32576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:59.119465 32576 hybrid_clock.cc:648] HybridClock initialized: now 1786545779119448 us; error 27 us; skew 500 ppm
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:59.121737 32576 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:42:59.122221 32576 webserver.cc:460] Webserver started at http://127.15.227.195:37809/ using document root <none> and password file <none>
I20260812 14:42:59.122416 32576 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:59.122462 32576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:59.122582 32576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:42:59.123538 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "675b98b3ab5048d4b8acd2f10ce69965"
format_stamp: "Formatted at 2026-08-12 14:42:59 on dist-test-slave-q9h9"
server_key: "43551846f1deefba98a677c5223f7bbb"
server_key_iv: "d9e5864e5aa7b1c547d4b80f1d6dee9e"
server_key_version: "encryptionkey@0"
I20260812 14:42:59.123855 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "675b98b3ab5048d4b8acd2f10ce69965"
format_stamp: "Formatted at 2026-08-12 14:42:59 on dist-test-slave-q9h9"
server_key: "43551846f1deefba98a677c5223f7bbb"
server_key_iv: "d9e5864e5aa7b1c547d4b80f1d6dee9e"
server_key_version: "encryptionkey@0"
I20260812 14:42:59.125036 32576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:42:59.125708 32608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.125861 32576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:42:59.125941 32576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "675b98b3ab5048d4b8acd2f10ce69965"
format_stamp: "Formatted at 2026-08-12 14:42:59 on dist-test-slave-q9h9"
server_key: "43551846f1deefba98a677c5223f7bbb"
server_key_iv: "d9e5864e5aa7b1c547d4b80f1d6dee9e"
server_key_version: "encryptionkey@0"
I20260812 14:42:59.126001 32576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:59.139683 32576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:59.141502 32576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:59.141628 32576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:59.141821 32576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:59.142129 32576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:59.142166 32576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.142194 32576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:59.142217 32576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.147675 32576 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35907
I20260812 14:42:59.147718 360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35907 every 8 connection(s)
I20260812 14:42:59.148000 32576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:42:59.154659 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 32576
I20260812 14:42:59.154747 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:42:59.154901 16271 external_mini_cluster.cc:1428] Setting key 697f326cdbf4c590b28c5def08155191
I20260812 14:42:59.156302 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.148632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:33619 (local address 127.15.227.254:37937)
0812 14:42:59.148682 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.148684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.148884 (+ 200us) server_negotiation.cc:407] Connection header received
0812 14:42:59.149350 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.149351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.149360 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.149410 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.150273 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.150757 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.151669 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.151838 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.151879 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.153766 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:59.153768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:59.153768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:59.153774 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:59.155042 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:59.155504 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:59.155505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:59.155505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:59.155568 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:59.155819 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:59.155819 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:59.155820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:59.155921 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:59.155969 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:59.156167 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 14:42:59.156231 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:42:59.156450 364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.148609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:33619)
0812 14:42:59.148764 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.148796 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.149269 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.149461 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.149464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.149987 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.150205 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.150210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.150819 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.150820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.151607 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.151609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.151889 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.151889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.151979 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.152388 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.152396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.153685 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:42:59.155119 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:59.155128 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:59.155130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:59.155432 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:59.155630 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:42:59.155630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:42:59.155631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:42:59.155761 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:42:59.155966 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:42:59.155969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:42:59.156071 (+ 102us) client_negotiation.cc:769] Sending connection context
0812 14:42:59.156146 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 14:42:59.156279 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":95,"threads_started":1}
I20260812 14:42:59.156880 361 heartbeater.cc:347] Connected to a master server at 127.15.227.254:37937
I20260812 14:42:59.156961 361 heartbeater.cc:464] Registering TS with master...
I20260812 14:42:59.157121 361 heartbeater.cc:511] Master 127.15.227.254:37937 requested a full tablet report, sending...
I20260812 14:42:59.157461 31941 ts_manager.cc:194] Registered new tserver with Master: 675b98b3ab5048d4b8acd2f10ce69965 (127.15.227.195:35907)
I20260812 14:42:59.157825 31941 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:33619
I20260812 14:42:59.166323 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab.
Entry for principal oryx/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab.
WARNING: no policy specified for oryx/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab.
Entry for principal oryx/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab.
WARNING: no policy specified for oryx/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab.
Entry for principal oryx/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab.
I20260812 14:42:59.225823 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 32080
I20260812 14:42:59.233302 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 32326
I20260812 14:42:59.242535 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 32576
I20260812 14:42:59.256412 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 31875
I20260812 14:42:59.268903 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37937
--webserver_interface=127.15.227.254
--webserver_port=43449
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:42:59.349773 381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:59.349956 381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:59.349987 381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:59.351126 381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:42:59.351171 381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:59.351190 381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:59.351207 381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:42:59.351222 381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:42:59.352666 381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=43449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:59.352897 381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:59.353107 381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:59.357566 398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.357645 388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.357743 392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:59.357779 381 server_base.cc:1034] running on GCE node
I20260812 14:42:59.357985 381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:59.358297 381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:59.359437 381 hybrid_clock.cc:648] HybridClock initialized: now 1786545779359426 us; error 28 us; skew 500 ppm
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545779, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:59.361662 381 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:42:59.362078 381 webserver.cc:460] Webserver started at http://127.15.227.254:43449/ using document root <none> and password file <none>
I20260812 14:42:59.362291 381 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:59.362355 381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:59.363636 381 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:42:59.364274 411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.364432 381 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:59.364526 381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "f54dee5234644ce5affe676d755cab93"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "1f4ee0590035583b30adb4f0f167d137"
server_key_iv: "b14e72857be2b0dd2fdb44aa677f0f03"
server_key_version: "encryptionkey@0"
I20260812 14:42:59.364809 381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:59.386415 381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:59.387955 381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:59.388079 381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:59.391705 381 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:37937
I20260812 14:42:59.391738 520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:37937 every 8 connection(s)
I20260812 14:42:59.392053 381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:42:59.393366 527 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:42:59.393411 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 381
I20260812 14:42:59.393653 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:38675
--local_ip_for_outbound_sockets=127.15.227.193
--tserver_master_addrs=127.15.227.254:37937
--webserver_port=35581
--webserver_interface=127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:42:59.394093 527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Bootstrap starting.
I20260812 14:42:59.402980 527 log.cc:824] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Log is configured to *not* fsync() on all Append() calls
I20260812 14:42:59.404453 527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:42:59.404697 527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Bootstrap complete.
I20260812 14:42:59.406791 527 raft_consensus.cc:348] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
I20260812 14:42:59.407006 527 raft_consensus.cc:729] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f54dee5234644ce5affe676d755cab93, State: Initialized, Role: FOLLOWER
I20260812 14:42:59.407145 527 consensus_queue.cc:260] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
I20260812 14:42:59.407218 527 raft_consensus.cc:388] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:42:59.407267 527 raft_consensus.cc:482] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:42:59.407312 527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:42:59.408278 527 raft_consensus.cc:504] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
I20260812 14:42:59.408373 527 leader_election.cc:302] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f54dee5234644ce5affe676d755cab93; no voters:
I20260812 14:42:59.408525 527 leader_election.cc:288] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:42:59.408586 537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:42:59.408771 527 sys_catalog.cc:564] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:42:59.408797 537 raft_consensus.cc:686] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [term 2 LEADER]: Becoming Leader. State: Replica: f54dee5234644ce5affe676d755cab93, State: Running, Role: LEADER
I20260812 14:42:59.408913 537 consensus_queue.cc:237] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } }
I20260812 14:42:59.409159 537 sys_catalog.cc:455] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f54dee5234644ce5affe676d755cab93. Latest consensus state: current_term: 2 leader_uuid: "f54dee5234644ce5affe676d755cab93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } } }
I20260812 14:42:59.409219 537 sys_catalog.cc:458] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:59.409351 542 sys_catalog.cc:455] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f54dee5234644ce5affe676d755cab93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f54dee5234644ce5affe676d755cab93" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37937 } } }
I20260812 14:42:59.409403 542 sys_catalog.cc:458] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93 [sys.catalog]: This master's current role is: LEADER
I20260812 14:42:59.409497 553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:42:59.409957 553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:42:59.410110 553 catalog_manager.cc:1259] Loaded cluster ID: 2e2837cb38f74d73b68ee6bfbcdf0b84
I20260812 14:42:59.410161 553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:42:59.411506 553 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:42:59.411686 553 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f54dee5234644ce5affe676d755cab93: Loaded TSK: 0
I20260812 14:42:59.411888 553 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:42:59.474749 535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:59.474942 535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:59.474965 535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:59.476094 535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:59.476132 535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:59.476157 535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:42:59.477537 535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.193.keytab
--principal=oryx/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:38675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=35581
--tserver_master_addrs=127.15.227.254:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:59.477725 535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:59.477938 535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:59.483503 577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.483526 579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:59.483722 535 server_base.cc:1034] running on GCE node
W20260812 14:42:59.483568 584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:59.484017 535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:59.484239 535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:59.485375 535 hybrid_clock.cc:648] HybridClock initialized: now 1786545779485363 us; error 31 us; skew 500 ppm
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545779, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:59.487922 535 init.cc:377] Logged in from keytab as oryx/127.15.227.193@KRBTEST.COM (short username oryx)
I20260812 14:42:59.488440 535 webserver.cc:460] Webserver started at http://127.15.227.193:35581/ using document root <none> and password file <none>
I20260812 14:42:59.488704 535 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:59.488785 535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:59.490471 535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:42:59.491104 593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.491281 535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:42:59.491359 535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "31e3eedd177b4c3bb3df464d68ecccf8"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "e32456aba3649074b63b2a27820ac1a6"
server_key_iv: "aedf5b45c4d0ebbeb1afb4c32bb124ed"
server_key_version: "encryptionkey@0"
I20260812 14:42:59.491647 535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:59.498832 535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:59.500622 535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:59.500731 535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:59.500912 535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:59.501174 535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:59.501206 535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.501225 535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:59.501258 535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.506819 535 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:38675
I20260812 14:42:59.506965 810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:38675 every 8 connection(s)
I20260812 14:42:59.507201 535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:42:59.509119 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 535
I20260812 14:42:59.509428 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:37083
--local_ip_for_outbound_sockets=127.15.227.194
--tserver_master_addrs=127.15.227.254:37937
--webserver_port=45967
--webserver_interface=127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.516008 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.507853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:46617)
0812 14:42:59.508011 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.508048 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.508516 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.508920 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.508924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.509512 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.509760 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.509765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.510949 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.510950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.512116 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.512119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.512931 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.512932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.513033 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.513469 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.513477 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.515815 (+ 2338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
W20260812 14:42:59.516237 811 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:37937 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.227.254:37937: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
W20260812 14:42:59.516871 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.508071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46617 (local address 127.15.227.254:37937)
0812 14:42:59.508294 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.508303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.508341 (+ 38us) server_negotiation.cc:407] Connection header received
0812 14:42:59.508692 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.508696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.508738 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.508847 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.509869 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.510848 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.512214 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.512837 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.512958 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.516695 (+ 3737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:46617: BlockingRecv error: recv got EOF from 127.15.227.193:46617 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":108,"threads_started":1}
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.522099 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.516446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:45409)
0812 14:42:59.517062 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.517072 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.517127 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.517306 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.517306 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.517442 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.517499 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.517500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.518410 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.518411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.519173 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.519175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.519625 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.519626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.519687 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.520094 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.520100 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.522012 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":67,"threads_started":1}
W20260812 14:42:59.522193 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.516502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:45409 (local address 127.15.227.254:37937)
0812 14:42:59.517014 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.517016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.517135 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:42:59.517187 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.517188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.517201 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.517292 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.517578 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.518318 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.519262 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.519547 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.519620 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.522130 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:45409: BlockingRecv error: recv got EOF from 127.15.227.193:45409 (error 108)
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":74,"threads_started":1}
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.527169 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.522563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:48945)
0812 14:42:59.522605 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.522614 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.522645 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.522759 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.522760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.522889 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.522919 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.522921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.523735 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.523736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.524530 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.524532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.524959 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.524960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.525011 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.525379 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.525383 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.527099 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 14:42:59.527264 811 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:42:59.527392 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.522620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:48945 (local address 127.15.227.254:37937)
0812 14:42:59.522652 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.522653 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.522662 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:42:59.522682 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.522683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.522691 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.522753 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.522986 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.523665 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.524604 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.524893 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.524955 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.527184 (+ 2229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:48945: BlockingRecv error: recv got EOF from 127.15.227.193:48945 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:42:59.595072 819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:59.595247 819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:59.595276 819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:59.596396 819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:59.596437 819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:59.596472 819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:42:59.597852 819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.194.keytab
--principal=oryx/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:37083
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=45967
--tserver_master_addrs=127.15.227.254:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:59.598086 819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:59.598287 819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:59.602950 832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.602959 830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.602959 835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:59.603215 819 server_base.cc:1034] running on GCE node
I20260812 14:42:59.603390 819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:59.603588 819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:59.604723 819 hybrid_clock.cc:648] HybridClock initialized: now 1786545779604712 us; error 26 us; skew 500 ppm
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545779, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:59.606988 819 init.cc:377] Logged in from keytab as oryx/127.15.227.194@KRBTEST.COM (short username oryx)
I20260812 14:42:59.607391 819 webserver.cc:460] Webserver started at http://127.15.227.194:45967/ using document root <none> and password file <none>
I20260812 14:42:59.607595 819 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:59.607662 819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:59.608867 819 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:59.609411 848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.609566 819 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.609644 819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "f2c16f82b1ee4b85ba3b7602db92e1ef"
format_stamp: "Formatted at 2026-08-12 14:42:58 on dist-test-slave-q9h9"
server_key: "5c304242eea9143ff0d8e667e32095a7"
server_key_iv: "52b8eeb6246e366c5234ed0e0a2d6362"
server_key_version: "encryptionkey@0"
I20260812 14:42:59.609905 819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:59.621682 819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:59.623207 819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:59.623313 819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:59.623464 819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:59.623706 819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:59.623739 819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.623757 819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:59.623780 819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.630434 819 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:37083
I20260812 14:42:59.630719 1054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:37083 every 8 connection(s)
I20260812 14:42:59.630851 819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:42:59.637154 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 819
I20260812 14:42:59.637414 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:35907
--local_ip_for_outbound_sockets=127.15.227.195
--tserver_master_addrs=127.15.227.254:37937
--webserver_port=37809
--webserver_interface=127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:39215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
W20260812 14:42:59.639276 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.632914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:36311 (local address 127.15.227.254:37937)
0812 14:42:59.633238 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.633240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.633262 (+ 22us) server_negotiation.cc:407] Connection header received
0812 14:42:59.633576 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.633577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.633594 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.633674 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.634736 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.635226 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.636155 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.636368 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.636425 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.639229 (+ 2804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:36311: BlockingRecv error: recv got EOF from 127.15.227.194:36311 (error 108)
Metrics: {"server-negotiator.queue_time_us":295}
I20260812 14:42:59.639334 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.632677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:36311)
0812 14:42:59.632860 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.632922 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.633453 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.633809 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.633812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.634393 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.634639 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.634645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.635294 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.635295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.636074 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.636076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.636437 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.636438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.636569 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.636967 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.636975 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.639159 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
W20260812 14:42:59.639559 1055 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:37937 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.227.254:37937: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.643791 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.639764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:34849)
0812 14:42:59.639870 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.639880 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.639928 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.640035 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.640035 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.640160 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.640206 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.640207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.640936 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.640937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.641403 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.641404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.641824 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.641825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.641868 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.642233 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.642237 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.643752 (+ 1515us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260812 14:42:59.643900 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.639826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:34849 (local address 127.15.227.254:37937)
0812 14:42:59.639866 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.639867 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.639915 (+ 48us) server_negotiation.cc:407] Connection header received
0812 14:42:59.639983 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.639983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.639991 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.640030 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.640266 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.640872 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.641477 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.641753 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.641826 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.643848 (+ 2022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:34849: BlockingRecv error: recv got EOF from 127.15.227.194:34849 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.647611 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.644003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:36159)
0812 14:42:59.644039 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.644047 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.644078 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.644187 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.644188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.644303 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.644327 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.644328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.644879 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.644880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.645339 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.645340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.645624 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.645625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.645673 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.646047 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.646051 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.647560 (+ 1509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 14:42:59.647732 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.644034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:36159 (local address 127.15.227.254:37937)
0812 14:42:59.644066 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.644068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.644076 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:42:59.644132 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.644133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.644142 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.644186 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.644386 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.644817 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.645403 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.645574 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.645622 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.647639 (+ 2017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:36159: BlockingRecv error: recv got EOF from 127.15.227.194:36159 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:42:59.647791 1055 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:42:59.718369 1058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:42:59.718536 1058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:42:59.718559 1058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:42:59.719671 1058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:42:59.719712 1058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:42:59.719740 1058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:42:59.721110 1058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:39215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/krb5kdc/oryx_127.15.227.195.keytab
--principal=oryx/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:35907
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=37809
--tserver_master_addrs=127.15.227.254:37937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:42:59.721315 1058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:42:59.721511 1058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:42:59.726104 1064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:59.726213 1058 server_base.cc:1034] running on GCE node
W20260812 14:42:59.726135 1067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:42:59.726143 1065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:42:59.726498 1058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:42:59.726725 1058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:42:59.727847 1058 hybrid_clock.cc:648] HybridClock initialized: now 1786545779727841 us; error 26 us; skew 500 ppm
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545779, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:42:59.729985 1058 init.cc:377] Logged in from keytab as oryx/127.15.227.195@KRBTEST.COM (short username oryx)
I20260812 14:42:59.730399 1058 webserver.cc:460] Webserver started at http://127.15.227.195:37809/ using document root <none> and password file <none>
I20260812 14:42:59.730604 1058 fs_manager.cc:359] Metadata directory not provided
I20260812 14:42:59.730659 1058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:42:59.731827 1058 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:42:59.732321 1082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.732445 1058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.732523 1058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "675b98b3ab5048d4b8acd2f10ce69965"
format_stamp: "Formatted at 2026-08-12 14:42:59 on dist-test-slave-q9h9"
server_key: "43551846f1deefba98a677c5223f7bbb"
server_key_iv: "d9e5864e5aa7b1c547d4b80f1d6dee9e"
server_key_version: "encryptionkey@0"
I20260812 14:42:59.732787 1058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:42:59.779181 1058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:42:59.780821 1058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:42:59.780946 1058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:42:59.781152 1058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:42:59.781508 1058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:42:59.781555 1058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.781585 1058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:42:59.781610 1058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:42:59.787104 1058 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35907
I20260812 14:42:59.787161 1307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35907 every 8 connection(s)
I20260812 14:42:59.787461 1058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.793339 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 1058
W20260812 14:42:59.794185 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.788081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:42471 (local address 127.15.227.254:37937)
0812 14:42:59.788124 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.788126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.788340 (+ 214us) server_negotiation.cc:407] Connection header received
0812 14:42:59.788785 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.788786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.788798 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.788833 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.789676 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.790194 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.791083 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.791263 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.791304 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.794125 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:42471: BlockingRecv error: recv got EOF from 127.15.227.195:42471 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:42:59.794572 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.788064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:42471)
0812 14:42:59.788215 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.788262 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.788694 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.788889 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.788892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.789415 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.789592 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.789597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.790265 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.790266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.791020 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.791022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.791315 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.791315 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.791408 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.791827 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.791836 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.794076 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
W20260812 14:42:59.794800 1308 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:37937 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.227.254:37937: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.799057 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.795008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:54163)
0812 14:42:59.795093 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.795101 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.795138 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.795258 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.795258 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.795392 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.795431 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.795433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.796002 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.796003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.796609 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.796611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.796941 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.796941 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.796996 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.797251 (+ 255us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.797256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.799018 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260812 14:42:59.799173 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.795054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:54163 (local address 127.15.227.254:37937)
0812 14:42:59.795094 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.795096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.795131 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:42:59.795181 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.795182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.795191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.795251 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.795489 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.795940 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.796675 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.796864 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.796920 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.799115 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:54163: BlockingRecv error: recv got EOF from 127.15.227.195:54163 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:42:59.800953 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.793815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.15.227.254:37937)
0812 14:42:59.793863 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.793865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.793911 (+ 46us) server_negotiation.cc:407] Connection header received
0812 14:42:59.793989 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.793990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.793999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.794084 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.794846 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.795509 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.796341 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.796524 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.796568 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.797881 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:42:59.797886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:42:59.797886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:42:59.797898 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:42:59.799650 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:59.800014 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:59.800017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:59.800017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:59.800136 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:42:59.800291 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:42:59.800292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:42:59.800293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:42:59.800560 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:42:59.800699 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 14:42:59.800763 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 14:42:59.800868 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:42:59 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:42:59.803251 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:42:59.799356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:55737)
0812 14:42:59.799405 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:42:59.799417 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:42:59.799458 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:42:59.799564 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:42:59.799565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:42:59.799689 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:42:59.799720 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.799722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.800465 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.800466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.801156 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:42:59.801159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.801477 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.801479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:42:59.801530 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.801850 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:42:59.801855 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:42:59.803211 (+ 1356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260812 14:42:59.803331 818 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:42:59.799398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:55737 (local address 127.15.227.254:37937)
0812 14:42:59.799434 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:42:59.799436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:42:59.799447 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:42:59.799511 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:42:59.799512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:42:59.799519 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:42:59.799557 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:42:59.799797 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.800381 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.801226 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:42:59.801413 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:42:59.801467 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:42:59.803288 (+ 1821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:55737: BlockingRecv error: recv got EOF from 127.15.227.195:55737 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:42:59.803421 1308 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 14:43:00 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:00 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:00.531607 1319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:00.527554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:49569)
0812 14:43:00.527674 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:00.527684 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:00.527729 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:00.527853 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:00.527854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:00.528002 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:00.528051 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:00.528052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.528657 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.528657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:00.529197 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:00.529199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.529477 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.529478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:00.529514 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:00.529876 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:00.529880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:00.531549 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":71,"threads_started":1}
W20260812 14:43:00.531704 1320 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:00.527619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:49569 (local address 127.15.227.254:37937)
0812 14:43:00.527745 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:00.527748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:00.527756 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:43:00.527777 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:00.527778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:00.527787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:00.527843 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:00.528108 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.528606 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.529254 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.529431 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.529473 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:00.531634 (+ 2161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:49569: BlockingRecv error: recv got EOF from 127.15.227.193:49569 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":65,"threads_started":1}
Aug 12 14:43:00 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:00 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:00.652606 1321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:00.648159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:35945)
0812 14:43:00.648279 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:00.648292 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:00.648363 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:00.648482 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:00.648483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:00.648665 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:00.648711 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:00.648713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.649372 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.649374 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:00.649915 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:00.649917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.650311 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.650312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:00.650367 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:00.650748 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:00.650752 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:00.652543 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
W20260812 14:43:00.652685 1320 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:00.648243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:35945 (local address 127.15.227.254:37937)
0812 14:43:00.648316 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:00.648318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:00.648343 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:43:00.648414 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:00.648415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:00.648428 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:00.648476 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:00.648789 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.649299 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.649993 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.650243 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.650293 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:00.652633 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:35945: BlockingRecv error: recv got EOF from 127.15.227.194:35945 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 14:43:00 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:00 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:00.808008 1327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:00.803753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:33059)
0812 14:43:00.803904 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:00.803915 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:00.803987 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:00.804114 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:00.804115 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:00.804287 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:00.804335 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:00.804337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.804963 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.804964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:00.805507 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:00.805508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.805835 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.805837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:00.805894 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:00.806300 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:00.806304 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:00.807944 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
W20260812 14:43:00.808091 1320 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:00.803821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:33059 (local address 127.15.227.254:37937)
0812 14:43:00.803885 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:00.803887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:00.803968 (+ 81us) server_negotiation.cc:407] Connection header received
0812 14:43:00.804038 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:00.804039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:00.804053 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:00.804100 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:00.804408 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.804890 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.805586 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:00.805773 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:00.805828 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:00.808032 (+ 2204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:33059: BlockingRecv error: recv got EOF from 127.15.227.195:33059 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 14:43:01 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:01 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:01.536358 1328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:01.531985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:46119)
0812 14:43:01.532108 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:01.532118 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:01.532175 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:01.532300 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:01.532301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:01.532451 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:01.532500 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:01.532502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.533226 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.533227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:01.533752 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:01.533755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.534075 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.534076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:01.534119 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:01.534557 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:01.534562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:01.536294 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
W20260812 14:43:01.536500 1331 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:01.532036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46119 (local address 127.15.227.254:37937)
0812 14:43:01.532145 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:01.532147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:01.532171 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:43:01.532218 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:01.532219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:01.532230 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:01.532289 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:01.532549 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.533168 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.533813 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.533999 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.534079 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:01.536389 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:46119: BlockingRecv error: recv got EOF from 127.15.227.193:46119 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
Aug 12 14:43:01 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:01 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:01.657310 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:01.653044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:53267)
0812 14:43:01.653196 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:01.653207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:01.653270 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:01.653392 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:01.653393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:01.653554 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:01.653603 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:01.653605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.654335 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.654336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:01.654875 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:01.654877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.655218 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.655219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:01.655275 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:01.655661 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:01.655665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:01.657225 (+ 1560us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
W20260812 14:43:01.657383 1331 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:01.653116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:53267 (local address 127.15.227.254:37937)
0812 14:43:01.653179 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:01.653181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:01.653253 (+ 72us) server_negotiation.cc:407] Connection header received
0812 14:43:01.653321 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:01.653322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:01.653333 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:01.653381 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:01.653685 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.654269 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.654951 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.655160 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.655214 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:01.657331 (+ 2117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:53267: BlockingRecv error: recv got EOF from 127.15.227.194:53267 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 14:43:01 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:01 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:01.814182 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:01.808422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:37749)
0812 14:43:01.808541 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:01.808604 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:01.808668 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:01.808820 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:01.808821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:01.809019 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:01.809091 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:01.809094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.809956 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.809957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:01.810827 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:01.810832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.811325 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.811327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:01.811391 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:01.811872 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:01.811955 (+ 83us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:01.814106 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
W20260812 14:43:01.814247 1331 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:01.808602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:37749 (local address 127.15.227.254:37937)
0812 14:43:01.808666 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:01.808668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:01.808676 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:43:01.808741 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:01.808742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:01.808754 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:01.808810 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:01.809159 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.809886 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.810931 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:01.811197 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:01.811265 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:01.814194 (+ 2929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:37749: BlockingRecv error: recv got EOF from 127.15.227.195:37749 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 12 14:43:02 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:02 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:02.540861 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:02.536726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:34711)
0812 14:43:02.536846 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:02.536858 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:02.536918 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:02.537044 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:02.537045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:02.537195 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:02.537246 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:02.537247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.537838 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.537839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:02.538366 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:02.538368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.538645 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.538645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:02.538680 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:02.539109 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:02.539114 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:02.540789 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
W20260812 14:43:02.540943 1336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:02.536800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:34711 (local address 127.15.227.254:37937)
0812 14:43:02.536930 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:02.536932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:02.536953 (+ 21us) server_negotiation.cc:407] Connection header received
0812 14:43:02.536972 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:02.536973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:02.536983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:02.537029 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:02.537295 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.537789 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.538421 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.538600 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.538640 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:02.540884 (+ 2244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:34711: BlockingRecv error: recv got EOF from 127.15.227.193:34711 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
Aug 12 14:43:02 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:02 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:02.662115 1339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:02.657752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:39537)
0812 14:43:02.657882 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:02.657893 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:02.657957 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:02.658098 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:02.658099 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:02.658265 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:02.658311 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:02.658314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.658929 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.658930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:02.659445 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:02.659447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.659773 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.659774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:02.659829 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:02.660270 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:02.660275 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:02.662005 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260812 14:43:02.662161 1336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:02.657819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:39537 (local address 127.15.227.254:37937)
0812 14:43:02.657878 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:02.657880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:02.657938 (+ 58us) server_negotiation.cc:407] Connection header received
0812 14:43:02.658009 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:02.658010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:02.658039 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:02.658084 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:02.658381 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.658858 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.659521 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.659710 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.659766 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:02.662115 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:39537: BlockingRecv error: recv got EOF from 127.15.227.194:39537 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 14:43:02 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:02 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:02.818712 1342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:02.814570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:48413)
0812 14:43:02.814685 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:02.814694 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:02.814755 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:02.814888 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:02.814889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:02.815044 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:02.815089 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:02.815091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.815663 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.815664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:02.816173 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:02.816174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.816442 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.816443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:02.816477 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:02.816865 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:02.816869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:02.818621 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
W20260812 14:43:02.818778 1336 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:02.814624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:48413 (local address 127.15.227.254:37937)
0812 14:43:02.814673 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:02.814675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:02.814740 (+ 65us) server_negotiation.cc:407] Connection header received
0812 14:43:02.814813 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:02.814814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:02.814830 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:02.814878 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:02.815134 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.815613 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.816222 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:02.816393 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:02.816435 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:02.818716 (+ 2281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:48413: BlockingRecv error: recv got EOF from 127.15.227.195:48413 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 14:43:03 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:03 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:03.545514 1345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:03.541320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:60937)
0812 14:43:03.541464 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:03.541474 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:03.541529 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:03.541661 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:03.541662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:03.541825 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:03.541873 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:03.541874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.542473 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.542473 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:03.542998 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:03.542999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.543360 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.543360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:03.543404 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:03.543847 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:03.543852 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:03.545456 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
W20260812 14:43:03.545612 1348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:03.541376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:60937 (local address 127.15.227.254:37937)
0812 14:43:03.541491 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:03.541493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:03.541514 (+ 21us) server_negotiation.cc:407] Connection header received
0812 14:43:03.541583 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:03.541584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:03.541594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:03.541649 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:03.541920 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.542424 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.543075 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.543305 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.543354 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:03.545544 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:60937: BlockingRecv error: recv got EOF from 127.15.227.193:60937 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":57,"threads_started":1}
Aug 12 14:43:03 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:03 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:03.672123 1349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:03.662529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:54697)
0812 14:43:03.662672 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:03.662684 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:03.662751 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:03.662904 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:03.662904 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:03.663073 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:03.663126 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:03.663128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.663713 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.663713 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:03.664245 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:03.664247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.664533 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.664534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:03.664587 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:03.670336 (+ 5749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:03.670341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:03.672067 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":59,"threads_started":1}
W20260812 14:43:03.672245 1348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:03.662634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:54697 (local address 127.15.227.254:37937)
0812 14:43:03.662687 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:03.662689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:03.662741 (+ 52us) server_negotiation.cc:407] Connection header received
0812 14:43:03.662807 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:03.662808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:03.662818 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:03.662862 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:03.663183 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.663659 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.664302 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.664470 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.664510 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:03.672178 (+ 7668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:54697: BlockingRecv error: recv got EOF from 127.15.227.194:54697 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 12 14:43:03 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:03 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:03.823421 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:03.819259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:54871)
0812 14:43:03.819383 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:03.819395 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:03.819465 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:03.819609 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:03.819610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:03.819770 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:03.819818 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:03.819820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.820451 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.820452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:03.820964 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:03.820966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.821285 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.821286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:03.821340 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:03.821726 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:03.821731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:03.823348 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
W20260812 14:43:03.823505 1348 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:03.819332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:54871 (local address 127.15.227.254:37937)
0812 14:43:03.819386 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:03.819389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:03.819453 (+ 64us) server_negotiation.cc:407] Connection header received
0812 14:43:03.819530 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:03.819531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:03.819545 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:03.819594 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:03.819886 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.820378 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.821035 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:03.821224 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:03.821278 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:03.823452 (+ 2174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:54871: BlockingRecv error: recv got EOF from 127.15.227.195:54871 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 14:43:04 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:04 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:04.550185 1351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:04.545937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:57605)
0812 14:43:04.546071 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:04.546080 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:04.546129 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:04.546261 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:04.546262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:04.546422 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:04.546470 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:04.546472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.547103 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.547104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:04.547641 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:04.547643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.548004 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.548005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:04.548044 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:04.548452 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:04.548457 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:04.550125 (+ 1668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
W20260812 14:43:04.550294 1352 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:04.546010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:57605 (local address 127.15.227.254:37937)
0812 14:43:04.546149 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:04.546151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:04.546159 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:43:04.546182 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:04.546182 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:04.546193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:04.546248 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:04.546538 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.547039 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.547738 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.547945 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.548006 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:04.550220 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:57605: BlockingRecv error: recv got EOF from 127.15.227.193:57605 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
Aug 12 14:43:04 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:04 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:04.676702 1353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:04.672510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:41791)
0812 14:43:04.672633 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:04.672643 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:04.672692 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:04.672820 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:04.672821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:04.672999 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:04.673050 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:04.673052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.673666 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.673666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:04.674242 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:04.674245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.674530 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.674530 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:04.674567 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:04.674968 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:04.674973 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:04.676642 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
W20260812 14:43:04.676791 1352 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:04.672625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:41791 (local address 127.15.227.254:37937)
0812 14:43:04.672679 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:04.672681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:04.672691 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:43:04.672747 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:04.672748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:04.672761 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:04.672808 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:04.673112 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.673598 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.674303 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.674483 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.674527 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:04.676731 (+ 2204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:41791: BlockingRecv error: recv got EOF from 127.15.227.194:41791 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 12 14:43:04 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:04 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:04.828158 1355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:04.823842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:60177)
0812 14:43:04.823981 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:04.823992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:04.824054 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:04.824189 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:04.824190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:04.824355 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:04.824423 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:04.824426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.825090 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.825091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:04.825618 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:04.825620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.825946 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.825947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:04.826005 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:04.826436 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:04.826441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:04.828082 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
W20260812 14:43:04.828228 1352 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:04.823900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:60177 (local address 127.15.227.254:37937)
0812 14:43:04.823959 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:04.823961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:04.824041 (+ 80us) server_negotiation.cc:407] Connection header received
0812 14:43:04.824111 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:04.824112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:04.824128 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:04.824177 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:04.824491 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.825021 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.825688 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:04.825888 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:04.825940 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:04.828175 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:60177: BlockingRecv error: recv got EOF from 127.15.227.195:60177 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 14:43:05 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:05 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:05.554924 1357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:05.550603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:60151)
0812 14:43:05.550732 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:05.550742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:05.550798 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:05.550916 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:05.550917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:05.551073 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:05.551122 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:05.551124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.551742 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.551743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:05.552282 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:05.552283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.552562 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.552562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:05.552596 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:05.553013 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:05.553018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:05.554846 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
W20260812 14:43:05.554996 1358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:05.550658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:60151 (local address 127.15.227.254:37937)
0812 14:43:05.550781 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:05.550783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:05.550792 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:43:05.550839 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:05.550840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:05.550850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:05.550906 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:05.551171 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.551690 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.552336 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.552518 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.552559 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:05.554934 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:60151: BlockingRecv error: recv got EOF from 127.15.227.193:60151 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
Aug 12 14:43:05 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:05 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:05.681605 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:05.677246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:48095)
0812 14:43:05.677386 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:05.677398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:05.677464 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:05.677588 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:05.677589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:05.677767 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:05.677815 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:05.677817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.678490 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.678491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:05.679032 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:05.679034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.679371 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.679372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:05.679428 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:05.679861 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:05.679865 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:05.681506 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
W20260812 14:43:05.681685 1358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:05.677324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:48095 (local address 127.15.227.254:37937)
0812 14:43:05.677386 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:05.677388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:05.677444 (+ 56us) server_negotiation.cc:407] Connection header received
0812 14:43:05.677515 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:05.677515 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:05.677528 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:05.677573 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:05.677887 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.678420 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.679114 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.679312 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.679366 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:05.681612 (+ 2246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:48095: BlockingRecv error: recv got EOF from 127.15.227.194:48095 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 14:43:05 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:05 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:05.832744 1360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:05.828574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:43847)
0812 14:43:05.828715 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:05.828726 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:05.828799 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:05.828907 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:05.828907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:05.829056 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:05.829104 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:05.829106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.829758 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.829759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:05.830306 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:05.830308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.830589 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.830590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:05.830624 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:05.830999 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:05.831003 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:05.832667 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
W20260812 14:43:05.832816 1358 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:05.828639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:43847 (local address 127.15.227.254:37937)
0812 14:43:05.828686 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:05.828689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:05.828784 (+ 95us) server_negotiation.cc:407] Connection header received
0812 14:43:05.828838 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:05.828839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:05.828850 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:05.828894 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:05.829190 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.829694 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.830363 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:05.830540 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:05.830586 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:05.832762 (+ 2176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:43847: BlockingRecv error: recv got EOF from 127.15.227.195:43847 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 14:43:06 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:06 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:06.559386 1367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:06.555346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:35981)
0812 14:43:06.555462 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:06.555473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:06.555534 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:06.555647 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:06.555648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:06.555801 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:06.555852 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:06.555854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.556448 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.556449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:06.556964 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:06.556966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.557240 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.557240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:06.557275 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:06.557654 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:06.557658 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:06.559329 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
W20260812 14:43:06.559500 1368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:06.555424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:35981 (local address 127.15.227.254:37937)
0812 14:43:06.555544 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:06.555546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:06.555553 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:43:06.555574 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:06.555574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:06.555585 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:06.555636 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:06.555911 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.556398 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.557017 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.557195 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.557235 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:06.559425 (+ 2190us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:35981: BlockingRecv error: recv got EOF from 127.15.227.193:35981 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
Aug 12 14:43:06 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:06 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:06.686719 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:06.681990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:47535)
0812 14:43:06.682161 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:06.682174 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:06.682243 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:06.682369 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:06.682370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:06.682545 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:06.682594 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:06.682598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.683223 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.683224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:06.683748 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:06.683749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.684103 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.684104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:06.684167 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:06.684680 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:06.684685 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:06.686635 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
W20260812 14:43:06.686930 1368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:06.682085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:47535 (local address 127.15.227.254:37937)
0812 14:43:06.682141 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:06.682144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:06.682233 (+ 89us) server_negotiation.cc:407] Connection header received
0812 14:43:06.682298 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:06.682299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:06.682313 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:06.682356 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:06.682667 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.683151 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.683824 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.684029 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.684090 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:06.686870 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:47535: BlockingRecv error: recv got EOF from 127.15.227.194:47535 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 12 14:43:06 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:06 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:06.837767 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:06.833227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:49399)
0812 14:43:06.833367 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:06.833378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:06.833435 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:06.833549 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:06.833550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:06.833706 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:06.833764 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:06.833765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.834449 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.834450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:06.834993 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:06.834995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.835369 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.835371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:06.835424 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:06.835881 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:06.835886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:06.837709 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
W20260812 14:43:06.837850 1368 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:06.833309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:49399 (local address 127.15.227.254:37937)
0812 14:43:06.833378 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:06.833379 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:06.833433 (+ 54us) server_negotiation.cc:407] Connection header received
0812 14:43:06.833486 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:06.833486 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:06.833500 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:06.833539 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:06.833824 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.834374 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.835055 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:06.835254 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:06.835330 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:06.837792 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:49399: BlockingRecv error: recv got EOF from 127.15.227.195:49399 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 14:43:07 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:07 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:07.563951 1372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:07.559780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:52575)
0812 14:43:07.559902 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:07.559912 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:07.559959 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:07.560089 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:07.560090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:07.560246 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:07.560296 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:07.560297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.560883 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.560884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:07.561446 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:07.561449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.561752 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.561753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:07.561816 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:07.562245 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:07.562250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:07.563893 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260812 14:43:07.564059 1373 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:07.559886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:52575 (local address 127.15.227.254:37937)
0812 14:43:07.559986 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:07.559988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:07.559995 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:43:07.560014 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:07.560015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:07.560025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:07.560080 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:07.560347 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.560829 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.561501 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.561677 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.561718 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:07.563991 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:52575: BlockingRecv error: recv got EOF from 127.15.227.193:52575 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":44,"threads_started":1}
Aug 12 14:43:07 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:07 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:07.691639 1374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:07.687368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:56011)
0812 14:43:07.687492 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:07.687504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:07.687570 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:07.687693 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:07.687694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:07.687863 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:07.687910 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:07.687912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.688667 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.688668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:07.689187 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:07.689189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.689507 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.689508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:07.689564 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:07.689972 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:07.689977 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:07.691551 (+ 1574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
W20260812 14:43:07.691706 1373 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:07.687439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:56011 (local address 127.15.227.254:37937)
0812 14:43:07.687502 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:07.687504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:07.687559 (+ 55us) server_negotiation.cc:407] Connection header received
0812 14:43:07.687621 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:07.687622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:07.687635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:07.687683 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:07.687980 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.688599 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.689267 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.689451 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.689502 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:07.691654 (+ 2152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:56011: BlockingRecv error: recv got EOF from 127.15.227.194:56011 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 12 14:43:07 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:07 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:07.842198 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:07.838158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:46223)
0812 14:43:07.838274 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:07.838284 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:07.838331 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:07.838459 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:07.838459 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:07.838617 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:07.838675 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:07.838676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.839285 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.839285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:07.839792 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:07.839794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.840056 (+ 262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.840056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:07.840090 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:07.840515 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:07.840520 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:07.842140 (+ 1620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
W20260812 14:43:07.842310 1373 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:07.838238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:46223 (local address 127.15.227.254:37937)
0812 14:43:07.838303 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:07.838305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:07.838317 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:43:07.838380 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:07.838381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:07.838392 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:07.838441 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:07.838723 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.839232 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.839844 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:07.840013 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:07.840054 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:07.842243 (+ 2189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:46223: BlockingRecv error: recv got EOF from 127.15.227.195:46223 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 14:43:08 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:08 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:08.568651 1378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:08.564373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:48163)
0812 14:43:08.564496 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:08.564506 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:08.564552 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:08.564674 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:08.564675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:08.564831 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:08.564880 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:08.564881 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.565476 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.565476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:08.565996 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:08.565998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.566327 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.566328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:08.566371 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:08.566779 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:08.566783 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:08.568564 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
W20260812 14:43:08.568728 1379 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:08.564432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:48163 (local address 127.15.227.254:37937)
0812 14:43:08.564566 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:08.564568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:08.564576 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:43:08.564599 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:08.564599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:08.564609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:08.564666 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:08.564931 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.565425 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.566087 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.566276 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.566319 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:08.568666 (+ 2347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:48163: BlockingRecv error: recv got EOF from 127.15.227.193:48163 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":72,"threads_started":1}
Aug 12 14:43:08 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:08 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:08.700645 1380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:08.692079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:44703)
0812 14:43:08.692217 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:08.692229 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:08.692295 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:08.692408 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:08.692409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:08.692583 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:08.692629 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:08.692631 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.693260 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.693262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:08.693861 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:08.693865 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.694257 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.694258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:08.694318 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:08.698708 (+ 4390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:08.698713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:08.700542 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
W20260812 14:43:08.700701 1379 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:08.692153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:44703 (local address 127.15.227.254:37937)
0812 14:43:08.692216 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:08.692217 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:08.692275 (+ 58us) server_negotiation.cc:407] Connection header received
0812 14:43:08.692344 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:08.692345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:08.692358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:08.692404 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:08.692699 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.693187 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.693938 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.694185 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.694236 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:08.700637 (+ 6401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:44703: BlockingRecv error: recv got EOF from 127.15.227.194:44703 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 14:43:08 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:08 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:08.846902 1386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:08.842601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:39765)
0812 14:43:08.842742 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:08.842753 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:08.842822 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:08.842946 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:08.842947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:08.843111 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:08.843157 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:08.843159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.843778 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.843780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:08.844300 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:08.844302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.844615 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.844616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:08.844670 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:08.845073 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:08.845077 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:08.846829 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
W20260812 14:43:08.846984 1379 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:08.842689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:39765 (local address 127.15.227.254:37937)
0812 14:43:08.842754 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:08.842756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:08.842812 (+ 56us) server_negotiation.cc:407] Connection header received
0812 14:43:08.842871 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:08.842872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:08.842886 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:08.842935 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:08.843227 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.843706 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.844374 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:08.844556 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:08.844609 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:08.846931 (+ 2322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:39765: BlockingRecv error: recv got EOF from 127.15.227.195:39765 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 14:43:09 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:09 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:09.573433 1388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:09.569077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:49947)
0812 14:43:09.569198 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:09.569209 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:09.569253 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:09.569366 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:09.569367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:09.569528 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:09.569579 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:09.569580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.570345 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.570346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:09.570868 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:09.570869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.571194 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.571194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:09.571228 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:09.571630 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:09.571634 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:09.573376 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
W20260812 14:43:09.573541 1389 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:09.569128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:49947 (local address 127.15.227.254:37937)
0812 14:43:09.569236 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:09.569238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:09.569246 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:43:09.569294 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:09.569294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:09.569304 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:09.569357 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:09.569699 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.570280 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.570930 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.571142 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.571191 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:09.573462 (+ 2271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:49947: BlockingRecv error: recv got EOF from 127.15.227.193:49947 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
Aug 12 14:43:09 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:09 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:09.705375 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:09.701053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:51367)
0812 14:43:09.701183 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:09.701194 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:09.701264 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:09.701390 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:09.701392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:09.701559 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:09.701606 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:09.701608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.702308 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.702310 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:09.702841 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:09.702843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.703173 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.703174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:09.703228 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:09.703657 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:09.703662 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:09.705282 (+ 1620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
W20260812 14:43:09.705453 1389 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:09.701113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:51367 (local address 127.15.227.254:37937)
0812 14:43:09.701186 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:09.701189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:09.701256 (+ 67us) server_negotiation.cc:407] Connection header received
0812 14:43:09.701314 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:09.701315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:09.701328 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:09.701376 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:09.701679 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.702236 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.702920 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.703112 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.703174 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:09.705393 (+ 2219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:51367: BlockingRecv error: recv got EOF from 127.15.227.194:51367 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 14:43:09 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:09 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:09.851724 1391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:09.847303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:47649)
0812 14:43:09.847428 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:09.847438 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:09.847483 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:09.847589 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:09.847590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:09.847743 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:09.847790 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:09.847792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.848521 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.848523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:09.849048 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:09.849050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.849376 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.849377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:09.849413 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:09.849823 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:09.849828 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:09.851632 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
W20260812 14:43:09.851784 1389 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:09.847421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:47649 (local address 127.15.227.254:37937)
0812 14:43:09.847482 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:09.847484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:09.847492 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:43:09.847516 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:09.847516 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:09.847528 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:09.847573 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:09.847857 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.848425 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.849131 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:09.849318 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:09.849373 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:09.851733 (+ 2360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:47649: BlockingRecv error: recv got EOF from 127.15.227.195:47649 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 14:43:10 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:10 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:10.578322 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:10.573829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:41503)
0812 14:43:10.573954 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:10.573965 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:10.574043 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:10.574174 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:10.574175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:10.574336 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:10.574384 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:10.574386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.575106 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.575108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:10.575741 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:10.575744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.576068 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.576068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:10.576130 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:10.576541 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:10.576546 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:10.578254 (+ 1708us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
W20260812 14:43:10.578409 1393 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:10.573906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:41503 (local address 127.15.227.254:37937)
0812 14:43:10.574001 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:10.574003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:10.574026 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:43:10.574098 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:10.574099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:10.574109 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:10.574160 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:10.574439 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.575002 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.575810 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.576012 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.576060 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:10.578339 (+ 2279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:41503: BlockingRecv error: recv got EOF from 127.15.227.193:41503 (error 108)
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":47,"threads_started":1}
Aug 12 14:43:10 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:10 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:10.710383 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:10.705915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:41659)
0812 14:43:10.706079 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:10.706090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:10.706156 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:10.706289 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:10.706290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:10.706466 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:10.706513 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:10.706515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.707150 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.707151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:10.707691 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:10.707693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.708011 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.708012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:10.708089 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:10.708488 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:10.708492 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:10.710289 (+ 1797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":69,"threads_started":1}
W20260812 14:43:10.710423 1393 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:10.705991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:41659 (local address 127.15.227.254:37937)
0812 14:43:10.706049 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:10.706051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:10.706144 (+ 93us) server_negotiation.cc:407] Connection header received
0812 14:43:10.706219 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:10.706219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:10.706233 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:10.706277 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:10.706587 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.707073 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.707762 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.707952 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.708007 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:10.710370 (+ 2363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:41659: BlockingRecv error: recv got EOF from 127.15.227.194:41659 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 12 14:43:10 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:10 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:10.856490 1396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:10.852177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:38309)
0812 14:43:10.852306 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:10.852316 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:10.852371 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:10.852485 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:10.852485 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:10.852650 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:10.852696 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:10.852697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.853285 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.853285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:10.853841 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:10.853843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.854202 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.854202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:10.854239 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:10.854664 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:10.854668 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:10.856419 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
W20260812 14:43:10.856563 1393 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:10.852234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:38309 (local address 127.15.227.254:37937)
0812 14:43:10.852305 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:10.852308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:10.852358 (+ 50us) server_negotiation.cc:407] Connection header received
0812 14:43:10.852410 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:10.852410 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:10.852423 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:10.852466 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:10.852746 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.853233 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.853924 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:10.854149 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:10.854201 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:10.856502 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:38309: BlockingRecv error: recv got EOF from 127.15.227.195:38309 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 12 14:43:11 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:11 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:11.582814 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:11.578727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:57235)
0812 14:43:11.578877 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:11.578889 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:11.578945 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:11.579057 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:11.579057 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:11.579229 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:11.579278 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:11.579280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.579867 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.579868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:11.580395 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:11.580396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.580671 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.580672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:11.580709 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:11.581097 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:11.581101 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:11.582757 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
W20260812 14:43:11.582919 1399 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:11.578811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:57235 (local address 127.15.227.254:37937)
0812 14:43:11.578941 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:11.578943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:11.578951 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:43:11.578989 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:11.578990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:11.578999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:11.579045 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:11.579330 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.579815 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.580448 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.580625 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.580666 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:11.582846 (+ 2180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:57235: BlockingRecv error: recv got EOF from 127.15.227.193:57235 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
Aug 12 14:43:11 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:11 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:11.715427 1402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:11.710872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:54531)
0812 14:43:11.711005 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:11.711015 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:11.711073 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:11.711246 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:11.711247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:11.711415 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:11.711465 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:11.711467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.712093 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.712095 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:11.712679 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:11.712682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.712985 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.712986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:11.713030 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:11.713471 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:11.713476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:11.715334 (+ 1858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260812 14:43:11.715488 1399 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:11.710935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:54531 (local address 127.15.227.254:37937)
0812 14:43:11.710989 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:11.710991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:11.711066 (+ 75us) server_negotiation.cc:407] Connection header received
0812 14:43:11.711123 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:11.711124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:11.711135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:11.711183 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:11.711523 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.712025 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.712752 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.712928 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.712967 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:11.715437 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:54531: BlockingRecv error: recv got EOF from 127.15.227.194:54531 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 12 14:43:11 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:11 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:11.862327 1403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:11.856870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:33957)
0812 14:43:11.857025 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:11.857037 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:11.857105 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:11.857235 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:11.857236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:11.857455 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:11.857516 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:11.857518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.858450 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.858451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:11.859310 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:11.859313 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.859744 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.859746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:11.859808 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:11.860284 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:11.860289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:11.862235 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":56,"threads_started":1}
W20260812 14:43:11.862411 1399 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:11.856914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:33957 (local address 127.15.227.254:37937)
0812 14:43:11.856962 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:11.856965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:11.857098 (+ 133us) server_negotiation.cc:407] Connection header received
0812 14:43:11.857158 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:11.857159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:11.857174 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:11.857227 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:11.857580 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.858359 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.859377 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:11.859678 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:11.859736 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:11.862344 (+ 2608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:33957: BlockingRecv error: recv got EOF from 127.15.227.195:33957 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 12 14:43:12 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:12 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:12.587492 1406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:12.583229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:47965)
0812 14:43:12.583384 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:12.583395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:12.583458 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:12.583577 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:12.583578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:12.583747 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:12.583796 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:12.583798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.584413 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.584414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:12.584973 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:12.584976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.585261 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.585262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:12.585301 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:12.585742 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:12.585747 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:12.587406 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
W20260812 14:43:12.587570 1407 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:12.583277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:47965 (local address 127.15.227.254:37937)
0812 14:43:12.583389 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:12.583391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:12.583442 (+ 51us) server_negotiation.cc:407] Connection header received
0812 14:43:12.583508 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:12.583509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:12.583520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:12.583563 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:12.583849 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.584334 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.585035 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.585215 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.585257 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:12.587498 (+ 2241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:47965: BlockingRecv error: recv got EOF from 127.15.227.193:47965 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
Aug 12 14:43:12 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:12 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:12.720146 1411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:12.715861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:58103)
0812 14:43:12.715993 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:12.716006 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:12.716070 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:12.716193 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:12.716194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:12.716373 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:12.716418 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:12.716420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.717043 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.717044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:12.717587 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:12.717589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.717970 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.717972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:12.718054 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:12.718458 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:12.718463 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:12.720071 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
W20260812 14:43:12.720221 1407 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:12.715962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:58103 (local address 127.15.227.254:37937)
0812 14:43:12.716016 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:12.716018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:12.716052 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:43:12.716123 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:12.716124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:12.716138 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:12.716181 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:12.716486 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.716972 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.717662 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.717902 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.717961 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:12.720169 (+ 2208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:58103: BlockingRecv error: recv got EOF from 127.15.227.194:58103 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 12 14:43:12 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:12 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:12.867164 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:12.862868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:35003)
0812 14:43:12.863036 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:12.863047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:12.863120 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:12.863254 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:12.863255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:12.863417 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:12.863466 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:12.863468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.864057 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.864057 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:12.864583 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:12.864585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.864860 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.864860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:12.864895 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:12.865312 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:12.865316 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:12.867085 (+ 1769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":89,"threads_started":1}
W20260812 14:43:12.867228 1407 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:12.862923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:35003 (local address 127.15.227.254:37937)
0812 14:43:12.862981 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:12.862983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:12.863109 (+ 126us) server_negotiation.cc:407] Connection header received
0812 14:43:12.863178 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:12.863178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:12.863192 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:12.863242 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:12.863521 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.864005 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.864639 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:12.864814 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:12.864855 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:12.867172 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:35003: BlockingRecv error: recv got EOF from 127.15.227.195:35003 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 12 14:43:13 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:13 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:13.591894 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:13.587878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:47639)
0812 14:43:13.588000 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:13.588009 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:13.588055 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:13.588174 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:13.588174 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:13.588337 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:13.588386 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:13.588387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.588980 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.588981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:13.589489 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:13.589490 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.589760 (+ 270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.589760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:13.589807 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:13.590245 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:13.590250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:13.591835 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
W20260812 14:43:13.591989 1415 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:13.587935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:47639 (local address 127.15.227.254:37937)
0812 14:43:13.588049 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:13.588051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:13.588060 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:43:13.588097 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:13.588097 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:13.588108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:13.588163 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:13.588438 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.588928 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.589540 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.589715 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.589755 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:13.591923 (+ 2168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:47639: BlockingRecv error: recv got EOF from 127.15.227.193:47639 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
Aug 12 14:43:13 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:13 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:13.728513 1417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:13.720567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:60125)
0812 14:43:13.720729 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:13.720740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:13.720802 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:13.720993 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:13.720994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:13.721154 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:13.721211 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:13.721213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.721855 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.721857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:13.722460 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:13.722463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.722788 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.722789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:13.722848 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:13.726728 (+ 3880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:13.726734 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:13.728441 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":56,"threads_started":1}
W20260812 14:43:13.728585 1415 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:13.720625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:60125 (local address 127.15.227.254:37937)
0812 14:43:13.720705 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:13.720706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:13.720796 (+ 90us) server_negotiation.cc:407] Connection header received
0812 14:43:13.720861 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:13.720862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:13.720874 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:13.720924 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:13.721283 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.721785 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.722530 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.722723 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.722785 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:13.728528 (+ 5743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:60125: BlockingRecv error: recv got EOF from 127.15.227.194:60125 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 12 14:43:13 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:13 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.195@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:13.872346 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:13.867667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.195:55141)
0812 14:43:13.867789 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:13.867802 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:13.867872 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:13.868015 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:13.868016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:13.868186 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:13.868231 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:13.868233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.868909 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.868911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:13.869659 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:13.869662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.869996 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.869997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:13.870070 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:13.870501 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:13.870506 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:13.872235 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
W20260812 14:43:13.872357 1415 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:13.867762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:55141 (local address 127.15.227.254:37937)
0812 14:43:13.867819 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:13.867821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:13.867862 (+ 41us) server_negotiation.cc:407] Connection header received
0812 14:43:13.867943 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:13.867944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:13.867957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:13.868003 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:13.868303 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.868829 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.869734 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:13.869938 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:13.869992 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:13.872311 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.195:55141: BlockingRecv error: recv got EOF from 127.15.227.195:55141 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 12 14:43:14 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:14 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.193@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:14.596645 1421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:14.592383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.193:55757)
0812 14:43:14.592507 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:14.592518 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:14.592567 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:14.592684 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:14.592684 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:14.592838 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:14.592888 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:14.592890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.593559 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.593560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:14.594117 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:14.594119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.594425 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.594426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:14.594462 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:14.594961 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:14.594965 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:14.596580 (+ 1615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":69,"threads_started":1}
W20260812 14:43:14.596731 1422 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:14.592456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:55757 (local address 127.15.227.254:37937)
0812 14:43:14.592556 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:14.592558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:14.592567 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:43:14.592609 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:14.592609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:14.592619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:14.592674 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:14.592948 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.593465 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.594185 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.594377 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.594420 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:14.596669 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.193:55757: BlockingRecv error: recv got EOF from 127.15.227.193:55757 (error 108)
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":51,"threads_started":1}
Aug 12 14:43:14 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
Aug 12 14:43:14 dist-test-slave-q9h9 krb5kdc[31848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.227.194@KRBTEST.COM for oryx/127.15.227.254@KRBTEST.COM, Server not found in Kerberos database
I20260812 14:43:14.732992 1423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:14.728971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:37937 (local address 127.15.227.194:44757)
0812 14:43:14.729108 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:14.729117 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:14.729169 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:14.729275 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:14.729276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:14.729422 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:14.729469 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:14.729470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.730081 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.730081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:14.730601 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:14.730603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.730942 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.730942 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:14.730979 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:14.731378 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:14.731382 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:14.732933 (+ 1551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.227.254:37937: Server oryx/127.15.227.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
W20260812 14:43:14.733081 1422 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:14.729036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:44757 (local address 127.15.227.254:37937)
0812 14:43:14.729090 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:14.729091 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:14.729164 (+ 73us) server_negotiation.cc:407] Connection header received
0812 14:43:14.729208 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:14.729209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:14.729219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:14.729266 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:14.729511 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.729988 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.730679 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.730887 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.730942 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:14.733022 (+ 2080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.227.194:44757: BlockingRecv error: recv got EOF from 127.15.227.194:44757 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:43:14.799968 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 535
I20260812 14:43:14.812386 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 819
I20260812 14:43:14.820609 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 1058
I20260812 14:43:14.837123 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 381
2026-08-12T14:43:14Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17607 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T14:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:43:14Z Disabled control of system clock
I20260812 14:43:14.868033 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:38475
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:38475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 14:43:14.945690 1434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:14.945870 1434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:14.945899 1434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:14.947122 1434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:43:14.947162 1434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:14.947173 1434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:14.947184 1434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:43:14.947194 1434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:43:14.948439 1434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:38475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:38475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:14.948623 1434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:14.948850 1434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:14.953423 1440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:14.953423 1447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:14.953533 1434 server_base.cc:1034] running on GCE node
W20260812 14:43:14.953455 1452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:14.953859 1434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:14.954151 1434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:14.955293 1434 hybrid_clock.cc:648] HybridClock initialized: now 1786545794955274 us; error 37 us; skew 500 ppm
I20260812 14:43:14.956368 1434 webserver.cc:460] Webserver started at http://127.15.227.254:35275/ using document root <none> and password file <none>
I20260812 14:43:14.956563 1434 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:14.956609 1434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:14.956712 1434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:14.957666 1434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "5a7c37ec47cc43159a362f644bcec232"
format_stamp: "Formatted at 2026-08-12 14:43:14 on dist-test-slave-q9h9"
server_key: "25493e4677f8feb9a154a2a3476cce5d"
server_key_iv: "5f637f6b8c335ae6985e872a73b361e1"
server_key_version: "encryptionkey@0"
I20260812 14:43:14.957978 1434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "5a7c37ec47cc43159a362f644bcec232"
format_stamp: "Formatted at 2026-08-12 14:43:14 on dist-test-slave-q9h9"
server_key: "25493e4677f8feb9a154a2a3476cce5d"
server_key_iv: "5f637f6b8c335ae6985e872a73b361e1"
server_key_version: "encryptionkey@0"
I20260812 14:43:14.959161 1434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:43:14.959810 1465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:14.959968 1434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:43:14.960053 1434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "5a7c37ec47cc43159a362f644bcec232"
format_stamp: "Formatted at 2026-08-12 14:43:14 on dist-test-slave-q9h9"
server_key: "25493e4677f8feb9a154a2a3476cce5d"
server_key_iv: "5f637f6b8c335ae6985e872a73b361e1"
server_key_version: "encryptionkey@0"
I20260812 14:43:14.960124 1434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:14.966560 1434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:14.966786 1434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:14.966902 1434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:14.970432 1434 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:38475
I20260812 14:43:14.970463 1548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:38475 every 8 connection(s)
I20260812 14:43:14.970724 1434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:43:14.971324 1550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:43:14.971598 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 1434
I20260812 14:43:14.971678 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:43:14.971858 16271 external_mini_cluster.cc:1428] Setting key 0f63146c5dd2d4938b7e88896d46e477
I20260812 14:43:14.974726 1550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232: Bootstrap starting.
I20260812 14:43:14.975310 1550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232: Neither blocks nor log segments found. Creating new log.
I20260812 14:43:14.975571 1550 log.cc:824] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232: Log is configured to *not* fsync() on all Append() calls
I20260812 14:43:14.976189 1550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232: No bootstrap required, opened a new log
I20260812 14:43:14.977008 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:14.972291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.15.227.254:38475)
0812 14:43:14.972499 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:14.972507 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:14.972537 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:43:14.972966 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:14.972975 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:14.973060 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:14.973200 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:14.973344 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.974659 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.975394 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:14.975814 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:14.975920 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:14.976462 (+ 542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:14.976466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:14.976467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:14.976615 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:14.976644 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:14.976653 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:14.976706 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:14.976776 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:43:14.976850 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260812 14:43:14.978107 1550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a7c37ec47cc43159a362f644bcec232" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38475 } }
I20260812 14:43:14.978235 1550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:43:14.978266 1550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7c37ec47cc43159a362f644bcec232, State: Initialized, Role: FOLLOWER
I20260812 14:43:14.978375 1550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a7c37ec47cc43159a362f644bcec232" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38475 } }
I20260812 14:43:14.978446 1550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:43:14.978475 1550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:43:14.978520 1550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:43:14.978987 1550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a7c37ec47cc43159a362f644bcec232" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38475 } }
I20260812 14:43:14.979079 1550 leader_election.cc:302] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a7c37ec47cc43159a362f644bcec232; no voters:
I20260812 14:43:14.979254 1550 leader_election.cc:288] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:43:14.979297 1558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:43:14.979444 1558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [term 1 LEADER]: Becoming Leader. State: Replica: 5a7c37ec47cc43159a362f644bcec232, State: Running, Role: LEADER
I20260812 14:43:14.979496 1550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:43:14.979589 1558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a7c37ec47cc43159a362f644bcec232" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38475 } }
I20260812 14:43:14.979920 1562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a7c37ec47cc43159a362f644bcec232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a7c37ec47cc43159a362f644bcec232" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38475 } } }
I20260812 14:43:14.979998 1562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [sys.catalog]: This master's current role is: LEADER
I20260812 14:43:14.980170 1564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a7c37ec47cc43159a362f644bcec232. Latest consensus state: current_term: 1 leader_uuid: "5a7c37ec47cc43159a362f644bcec232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a7c37ec47cc43159a362f644bcec232" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38475 } } }
I20260812 14:43:14.980212 1575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:43:14.980716 1575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:43:14.980232 1564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232 [sys.catalog]: This master's current role is: LEADER
I20260812 14:43:14.982108 1575 catalog_manager.cc:1347] Generated new cluster ID: f3d4a8f64d164704862df506215c20a7
I20260812 14:43:14.982161 1575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:43:14.991742 1575 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:43:14.992187 1575 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:43:14.996964 1575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a7c37ec47cc43159a362f644bcec232: Generated new TSK 0
I20260812 14:43:14.997092 1575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:43:15.002558 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38475
--builtin_ntp_servers=127.15.227.212:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 14:43:15.084795 1590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:15.084980 1590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:15.085009 1590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:15.086122 1590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:15.086156 1590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:15.086189 1590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:43:15.087471 1590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:15.087689 1590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:15.087953 1590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:15.093654 1599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.093695 1600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.093914 1604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.093899 1590 server_base.cc:1034] running on GCE node
I20260812 14:43:15.094134 1590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:15.094368 1590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:15.095506 1590 hybrid_clock.cc:648] HybridClock initialized: now 1786545795095494 us; error 31 us; skew 500 ppm
I20260812 14:43:15.096676 1590 webserver.cc:460] Webserver started at http://127.15.227.193:38337/ using document root <none> and password file <none>
I20260812 14:43:15.096913 1590 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:15.096962 1590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:15.097088 1590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:15.098129 1590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "94b6a0f0044a436fb2527e5400fdc25c"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "70588a08a3fee13f1f14fb52837e406f"
server_key_iv: "4880483e02042ffbd2b96e2565959f6a"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.098471 1590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "94b6a0f0044a436fb2527e5400fdc25c"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "70588a08a3fee13f1f14fb52837e406f"
server_key_iv: "4880483e02042ffbd2b96e2565959f6a"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.099725 1590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:43:15.100452 1622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.100597 1590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:43:15.100678 1590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "94b6a0f0044a436fb2527e5400fdc25c"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "70588a08a3fee13f1f14fb52837e406f"
server_key_iv: "4880483e02042ffbd2b96e2565959f6a"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.100756 1590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:15.109294 1590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:15.109529 1590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:15.109642 1590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:15.109826 1590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:15.110152 1590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:15.110186 1590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.110212 1590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:15.110237 1590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.115590 1590 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:34963
I20260812 14:43:15.115675 1833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:34963 every 8 connection(s)
I20260812 14:43:15.115927 1590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:43:15.118896 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 1590
I20260812 14:43:15.119001 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:43:15.119196 16271 external_mini_cluster.cc:1428] Setting key 5a72a02289d4cb15353ed178a9546a45
I20260812 14:43:15.120196 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38475
--builtin_ntp_servers=127.15.227.212:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 14:43:15.120965 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.116619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59793 (local address 127.15.227.254:38475)
0812 14:43:15.116678 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.116680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.116879 (+ 199us) server_negotiation.cc:407] Connection header received
0812 14:43:15.117369 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.117370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.117382 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.117432 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.118037 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.118532 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.119743 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.120005 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.120072 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:15.120716 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:15.120717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:15.120718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:15.120722 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:15.120739 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:15.120743 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:15.120794 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:15.120853 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:43:15.120884 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:43:15.121084 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.116609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:38475 (local address 127.15.227.193:59793)
0812 14:43:15.116783 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:15.116812 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:15.117290 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:15.117479 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:15.117482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:15.117491 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:15.117930 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:15.117934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.118608 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.118609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:15.119672 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:15.119675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.120081 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.120082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:15.120190 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:15.120589 (+ 399us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:43:15.120589 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:15.120614 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:43:15.120634 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:15.120800 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:15.120802 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:15.120805 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:43:15.120847 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:43:15.120919 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260812 14:43:15.122292 1834 heartbeater.cc:347] Connected to a master server at 127.15.227.254:38475
I20260812 14:43:15.122380 1834 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:15.122547 1834 heartbeater.cc:511] Master 127.15.227.254:38475 requested a full tablet report, sending...
I20260812 14:43:15.123055 1495 ts_manager.cc:194] Registered new tserver with Master: 94b6a0f0044a436fb2527e5400fdc25c (127.15.227.193:34963)
I20260812 14:43:15.123859 1495 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.193:59793
W20260812 14:43:15.200773 1842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:15.200941 1842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:15.200960 1842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:15.201988 1842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:15.202037 1842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:15.202080 1842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:43:15.203361 1842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:15.203560 1842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:15.203815 1842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:15.208444 1853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.208431 1850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.208622 1842 server_base.cc:1034] running on GCE node
W20260812 14:43:15.208711 1849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.208910 1842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:15.209124 1842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:15.210260 1842 hybrid_clock.cc:648] HybridClock initialized: now 1786545795210254 us; error 32 us; skew 500 ppm
I20260812 14:43:15.211304 1842 webserver.cc:460] Webserver started at http://127.15.227.194:37199/ using document root <none> and password file <none>
I20260812 14:43:15.211513 1842 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:15.211560 1842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:15.211681 1842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:15.212604 1842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "7da39946d00042ac9d56577d0bf2bd43"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "5d3045d5a4d38a3fcaf5a40a4d8c470b"
server_key_iv: "30a242cf5084165cfb099a95669020c5"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.212905 1842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "7da39946d00042ac9d56577d0bf2bd43"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "5d3045d5a4d38a3fcaf5a40a4d8c470b"
server_key_iv: "30a242cf5084165cfb099a95669020c5"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.214128 1842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:43:15.214813 1860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.214972 1842 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:43:15.215050 1842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "7da39946d00042ac9d56577d0bf2bd43"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "5d3045d5a4d38a3fcaf5a40a4d8c470b"
server_key_iv: "30a242cf5084165cfb099a95669020c5"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.215116 1842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:15.224772 1842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:15.225003 1842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:15.225116 1842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:15.225306 1842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:15.225592 1842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:15.225636 1842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.225670 1842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:15.225694 1842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.231145 1842 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:36367
I20260812 14:43:15.231201 2069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:36367 every 8 connection(s)
I20260812 14:43:15.231484 1842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:43:15.235833 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 1842
I20260812 14:43:15.235922 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:43:15.236114 16271 external_mini_cluster.cc:1428] Setting key 771a6fff8ef9a015e0df8e2067a66d21
I20260812 14:43:15.236606 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.232129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:54001 (local address 127.15.227.254:38475)
0812 14:43:15.232187 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.232189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.232384 (+ 195us) server_negotiation.cc:407] Connection header received
0812 14:43:15.232877 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.232879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.232893 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.232960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.233605 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.234172 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.235162 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.235374 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.235439 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:15.236299 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:15.236301 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:15.236302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:15.236309 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:15.236328 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:15.236333 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:15.236391 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:15.236495 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:43:15.236522 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:43:15.236725 2073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.232111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:38475 (local address 127.15.227.194:54001)
0812 14:43:15.232276 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:15.232306 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:15.232775 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:15.233025 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:15.233029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:15.233045 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:15.233505 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:15.233510 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.234278 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.234279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:15.235072 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:15.235074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.235454 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.235455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:15.235576 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:15.236088 (+ 512us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:43:15.236088 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:15.236123 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:43:15.236151 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:15.236408 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:15.236415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:15.236418 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:43:15.236481 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:43:15.236564 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
I20260812 14:43:15.237139 2071 heartbeater.cc:347] Connected to a master server at 127.15.227.254:38475
I20260812 14:43:15.237167 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38475
--builtin_ntp_servers=127.15.227.212:40697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 14:43:15.237241 2071 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:15.237419 2071 heartbeater.cc:511] Master 127.15.227.254:38475 requested a full tablet report, sending...
I20260812 14:43:15.237771 1495 ts_manager.cc:194] Registered new tserver with Master: 7da39946d00042ac9d56577d0bf2bd43 (127.15.227.194:36367)
I20260812 14:43:15.238180 1495 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.194:54001
W20260812 14:43:15.316644 2074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:15.316838 2074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:15.316864 2074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:15.317991 2074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:15.318050 2074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:15.318084 2074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:43:15.319428 2074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:40697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:15.319649 2074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:15.319921 2074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:15.324635 2088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.324635 2084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.324643 2086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.324724 2074 server_base.cc:1034] running on GCE node
I20260812 14:43:15.325047 2074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:15.325274 2074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:15.326408 2074 hybrid_clock.cc:648] HybridClock initialized: now 1786545795326393 us; error 35 us; skew 500 ppm
I20260812 14:43:15.327509 2074 webserver.cc:460] Webserver started at http://127.15.227.195:41429/ using document root <none> and password file <none>
I20260812 14:43:15.327718 2074 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:15.327761 2074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:15.327883 2074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:15.328836 2074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "40f29790852c42aaae8884cd7fea0400"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "abbf381a0ab19c55ca8dc3a3ef98c1d1"
server_key_iv: "23b9acc18e7b1047a0b32c913d5a278d"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.329140 2074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "40f29790852c42aaae8884cd7fea0400"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "abbf381a0ab19c55ca8dc3a3ef98c1d1"
server_key_iv: "23b9acc18e7b1047a0b32c913d5a278d"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.330325 2074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:43:15.330924 2098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.331069 2074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:43:15.331149 2074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "40f29790852c42aaae8884cd7fea0400"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "abbf381a0ab19c55ca8dc3a3ef98c1d1"
server_key_iv: "23b9acc18e7b1047a0b32c913d5a278d"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.331205 2074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:15.339169 2074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:15.339385 2074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:15.339496 2074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:15.339665 2074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:15.339937 2074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:15.339969 2074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.339995 2074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:15.340018 2074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.345228 2074 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:42153
I20260812 14:43:15.345274 2313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:42153 every 8 connection(s)
I20260812 14:43:15.345534 2074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:43:15.350124 1555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.346197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:46709 (local address 127.15.227.254:38475)
0812 14:43:15.346249 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.346251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.346454 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:43:15.346874 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.346876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.346887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.346949 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.347505 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.347984 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.348830 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.349003 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.349077 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:15.349793 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:15.349795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:15.349796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:15.349801 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:15.349821 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:15.349826 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:15.349885 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:15.349970 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:43:15.350001 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:43:15.350383 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.346187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:38475 (local address 127.15.227.195:46709)
0812 14:43:15.346363 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:15.346391 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:15.346790 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:15.346996 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:15.346999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:15.347007 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:15.347444 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:15.347448 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.348051 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.348052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:15.348772 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:15.348774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:15.349080 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:15.349080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:15.349171 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:15.349633 (+ 462us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:43:15.349633 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:15.349667 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:43:15.349691 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:15.349898 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:15.349903 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:15.349905 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:43:15.349959 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:43:15.350081 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260812 14:43:15.350901 2316 heartbeater.cc:347] Connected to a master server at 127.15.227.254:38475
I20260812 14:43:15.350987 2316 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:15.351169 2316 heartbeater.cc:511] Master 127.15.227.254:38475 requested a full tablet report, sending...
I20260812 14:43:15.352044 1495 ts_manager.cc:194] Registered new tserver with Master: 40f29790852c42aaae8884cd7fea0400 (127.15.227.195:42153)
I20260812 14:43:15.352330 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 2074
I20260812 14:43:15.352432 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:43:15.352532 1495 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.195:46709
I20260812 14:43:15.352628 16271 external_mini_cluster.cc:1428] Setting key 81951230209bb67fe0a7e989c5b2ebfb
I20260812 14:43:15.353456 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:43:15.355166 1555 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:15.354762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53774 (local address 127.15.227.254:38475)
0812 14:43:15.354805 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.354807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.354923 (+ 116us) server_negotiation.cc:407] Connection header received
0812 14:43:15.354979 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.354980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.354987 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.355029 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.355127 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53774: BlockingRecv error: recv got EOF from 127.0.0.1:53774 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:43:15.355252 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 1590
I20260812 14:43:15.371917 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 1842
I20260812 14:43:15.385879 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 2074
I20260812 14:43:15.402730 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 1434
2026-08-12T14:43:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (569 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T14:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:43:15Z Disabled control of system clock
I20260812 14:43:15.437337 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33337
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:43:15.515492 2342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:15.515674 2342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:15.515702 2342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:15.516786 2342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:43:15.516824 2342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:15.516841 2342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:15.516856 2342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:43:15.516872 2342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:43:15.518210 2342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33337
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:15.518431 2342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:15.518680 2342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:15.523161 2351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.523182 2353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.523247 2360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.523375 2342 server_base.cc:1034] running on GCE node
I20260812 14:43:15.523613 2342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:15.523856 2342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:15.524989 2342 hybrid_clock.cc:648] HybridClock initialized: now 1786545795524972 us; error 33 us; skew 500 ppm
I20260812 14:43:15.526124 2342 webserver.cc:460] Webserver started at http://127.15.227.254:41509/ using document root <none> and password file <none>
I20260812 14:43:15.526360 2342 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:15.526402 2342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:15.526520 2342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:15.527474 2342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "47dc8541374647bda78e9040c87a7711"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "3374f32d5782aa4eabad94d4277a896b"
server_key_iv: "6bbdd34d11df24ff18539cdeedc52227"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.527796 2342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "47dc8541374647bda78e9040c87a7711"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "3374f32d5782aa4eabad94d4277a896b"
server_key_iv: "6bbdd34d11df24ff18539cdeedc52227"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.528957 2342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:43:15.529584 2369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.529713 2342 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:43:15.529795 2342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "47dc8541374647bda78e9040c87a7711"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "3374f32d5782aa4eabad94d4277a896b"
server_key_iv: "6bbdd34d11df24ff18539cdeedc52227"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.529853 2342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:15.532939 2342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:15.533161 2342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:15.533286 2342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:15.536962 2342 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:33337
I20260812 14:43:15.536991 2468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:33337 every 8 connection(s)
I20260812 14:43:15.537279 2342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:43:15.537879 2469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:43:15.540297 2469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711: Bootstrap starting.
I20260812 14:43:15.540791 2469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711: Neither blocks nor log segments found. Creating new log.
I20260812 14:43:15.541039 2469 log.cc:824] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711: Log is configured to *not* fsync() on all Append() calls
I20260812 14:43:15.541113 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 2342
I20260812 14:43:15.541199 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:43:15.541415 16271 external_mini_cluster.cc:1428] Setting key 195ed9077da880648187befe0d50a341
I20260812 14:43:15.541699 2469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711: No bootstrap required, opened a new log
I20260812 14:43:15.543671 2469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dc8541374647bda78e9040c87a7711" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33337 } }
I20260812 14:43:15.543839 2469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:43:15.543886 2469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47dc8541374647bda78e9040c87a7711, State: Initialized, Role: FOLLOWER
I20260812 14:43:15.544015 2469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dc8541374647bda78e9040c87a7711" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33337 } }
I20260812 14:43:15.544098 2469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:43:15.544127 2469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:43:15.544170 2469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:43:15.544783 2469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dc8541374647bda78e9040c87a7711" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33337 } }
I20260812 14:43:15.544895 2469 leader_election.cc:302] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47dc8541374647bda78e9040c87a7711; no voters:
I20260812 14:43:15.544914 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.541824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39190 (local address 127.15.227.254:33337)
0812 14:43:15.542001 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.542010 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.542071 (+ 61us) server_negotiation.cc:407] Connection header received
0812 14:43:15.543640 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.543649 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.543710 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.543859 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.544463 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:15.544467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:15.544467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:15.544471 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:15.544507 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:15.544518 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:15.544569 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:15.544646 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:43:15.544691 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260812 14:43:15.545179 2469 leader_election.cc:288] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:43:15.545434 2469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:43:15.545461 2478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:43:15.545565 2478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [term 1 LEADER]: Becoming Leader. State: Replica: 47dc8541374647bda78e9040c87a7711, State: Running, Role: LEADER
I20260812 14:43:15.545650 2478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dc8541374647bda78e9040c87a7711" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33337 } }
I20260812 14:43:15.545982 2480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47dc8541374647bda78e9040c87a7711. Latest consensus state: current_term: 1 leader_uuid: "47dc8541374647bda78e9040c87a7711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dc8541374647bda78e9040c87a7711" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33337 } } }
I20260812 14:43:15.546095 2480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [sys.catalog]: This master's current role is: LEADER
I20260812 14:43:15.545997 2479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47dc8541374647bda78e9040c87a7711" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47dc8541374647bda78e9040c87a7711" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33337 } } }
I20260812 14:43:15.546300 2479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711 [sys.catalog]: This master's current role is: LEADER
I20260812 14:43:15.546387 2495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:43:15.546891 2495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:43:15.548301 2495 catalog_manager.cc:1347] Generated new cluster ID: 082c9e0565ff448d92d79bab9251c944
I20260812 14:43:15.548343 2495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:43:15.556221 2495 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:43:15.556353 2495 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:43:15.560595 2495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47dc8541374647bda78e9040c87a7711: Generated new TSK 0
I20260812 14:43:15.560753 2495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:43:15.563035 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33337
--builtin_ntp_servers=127.15.227.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 14:43:15.679742 2515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:15.680001 2515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:15.680034 2515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:15.681612 2515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:15.681671 2515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:15.681720 2515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:43:15.683657 2515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:15.683905 2515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:15.684242 2515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:15.689404 2528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.689384 2526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.689568 2522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.689457 2515 server_base.cc:1034] running on GCE node
I20260812 14:43:15.689774 2515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:15.689998 2515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:15.691172 2515 hybrid_clock.cc:648] HybridClock initialized: now 1786545795691159 us; error 29 us; skew 500 ppm
I20260812 14:43:15.692332 2515 webserver.cc:460] Webserver started at http://127.15.227.193:41931/ using document root <none> and password file <none>
I20260812 14:43:15.692533 2515 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:15.692579 2515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:15.692705 2515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:15.693656 2515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "1e5e370a93904be984877acfb05dcd00"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "3d7c30c7a9b797dc975b9f2dd1ea5322"
server_key_iv: "7fa401d7d5a510ecd45d236aa2616ec8"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.693984 2515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "1e5e370a93904be984877acfb05dcd00"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "3d7c30c7a9b797dc975b9f2dd1ea5322"
server_key_iv: "7fa401d7d5a510ecd45d236aa2616ec8"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.695255 2515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:43:15.695951 2538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.696096 2515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.696178 2515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "1e5e370a93904be984877acfb05dcd00"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "3d7c30c7a9b797dc975b9f2dd1ea5322"
server_key_iv: "7fa401d7d5a510ecd45d236aa2616ec8"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.696246 2515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:15.699388 2515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:15.699609 2515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:15.699723 2515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:15.699918 2515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:15.700189 2515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:15.700234 2515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.700259 2515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:15.700281 2515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.705708 2515 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:42757
I20260812 14:43:15.705762 2737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:42757 every 8 connection(s)
I20260812 14:43:15.706048 2515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:43:15.708820 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 2515
I20260812 14:43:15.708913 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:43:15.709089 16271 external_mini_cluster.cc:1428] Setting key 17561aed839dbdf6bd71b507fbc07908
I20260812 14:43:15.709447 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.706670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:47651 (local address 127.15.227.254:33337)
0812 14:43:15.706715 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.706717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.706939 (+ 222us) server_negotiation.cc:407] Connection header received
0812 14:43:15.707444 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.707445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.707452 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.707490 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.708611 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:15.708613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:15.708613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:15.708616 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:15.708631 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:15.708634 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:15.708668 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:15.709356 (+ 688us) server_negotiation.cc:299] Negotiation successful
0812 14:43:15.709375 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:43:15.709826 2741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.706652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33337 (local address 127.15.227.193:47651)
0812 14:43:15.706817 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:15.706854 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:15.707366 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:15.707548 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:15.707552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:15.707561 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:15.708461 (+ 900us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:43:15.708461 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:15.708495 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:43:15.708526 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:15.709246 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:15.709253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:15.709256 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:43:15.709568 (+ 312us) client_negotiation.cc:240] Negotiation successful
0812 14:43:15.709627 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":81,"threads_started":1}
I20260812 14:43:15.710069 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33337
--builtin_ntp_servers=127.15.227.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 14:43:15.710292 2738 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33337
I20260812 14:43:15.710388 2738 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:15.710532 2738 heartbeater.cc:511] Master 127.15.227.254:33337 requested a full tablet report, sending...
I20260812 14:43:15.711033 2394 ts_manager.cc:194] Registered new tserver with Master: 1e5e370a93904be984877acfb05dcd00 (127.15.227.193:42757)
W20260812 14:43:15.790148 2744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:15.790350 2744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:15.790382 2744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:15.791492 2744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:15.791529 2744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:15.791561 2744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:43:15.792856 2744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:15.793082 2744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:15.793370 2744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:15.797873 2763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.797883 2753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.798097 2744 server_base.cc:1034] running on GCE node
W20260812 14:43:15.798264 2754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.798519 2744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:15.798703 2744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:15.799840 2744 hybrid_clock.cc:648] HybridClock initialized: now 1786545795799829 us; error 29 us; skew 500 ppm
I20260812 14:43:15.800978 2744 webserver.cc:460] Webserver started at http://127.15.227.194:43263/ using document root <none> and password file <none>
I20260812 14:43:15.801170 2744 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:15.801211 2744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:15.801304 2744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:15.802258 2744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "4a076b8d6cdc4a40a11363de5ce7f8e4"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "7602623633e1ac25adeeda168514533c"
server_key_iv: "2410f843590999053570374eea3876c0"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.802526 2744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "4a076b8d6cdc4a40a11363de5ce7f8e4"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "7602623633e1ac25adeeda168514533c"
server_key_iv: "2410f843590999053570374eea3876c0"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.803614 2744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:43:15.804211 2772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.804363 2744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:43:15.804440 2744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "4a076b8d6cdc4a40a11363de5ce7f8e4"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "7602623633e1ac25adeeda168514533c"
server_key_iv: "2410f843590999053570374eea3876c0"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.804490 2744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:15.807547 2744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:15.807742 2744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:15.807832 2744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:15.807967 2744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:15.808187 2744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:15.808214 2744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.808247 2744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:15.808264 2744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.813354 2744 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:36317
I20260812 14:43:15.813412 2994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:36317 every 8 connection(s)
I20260812 14:43:15.813681 2744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:43:15.815534 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 2744
I20260812 14:43:15.815635 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:43:15.815845 16271 external_mini_cluster.cc:1428] Setting key 5c28481c19cb860f87c4f03caf3e7916
I20260812 14:43:15.816417 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.814324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:40493 (local address 127.15.227.254:33337)
0812 14:43:15.814392 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.814394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.814569 (+ 175us) server_negotiation.cc:407] Connection header received
0812 14:43:15.814988 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.814990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.814996 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.815047 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.816193 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:15.816195 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:15.816195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:15.816200 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:15.816220 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:15.816225 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:15.816282 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:15.816338 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:43:15.816351 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 14:43:15.816571 2998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.814271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33337 (local address 127.15.227.194:40493)
0812 14:43:15.814461 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:15.814499 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:15.814896 (+ 397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:15.815108 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:15.815111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:15.815119 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:15.816045 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:43:15.816045 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:15.816078 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:43:15.816112 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:15.816283 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:15.816287 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:15.816289 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:43:15.816343 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:43:15.816387 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260812 14:43:15.816860 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33337
--builtin_ntp_servers=127.15.227.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 14:43:15.816991 2995 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33337
I20260812 14:43:15.817081 2995 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:15.817212 2995 heartbeater.cc:511] Master 127.15.227.254:33337 requested a full tablet report, sending...
I20260812 14:43:15.817533 2394 ts_manager.cc:194] Registered new tserver with Master: 4a076b8d6cdc4a40a11363de5ce7f8e4 (127.15.227.194:36317)
W20260812 14:43:15.895757 2999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:15.895939 2999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:15.895959 2999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:15.897037 2999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:15.897068 2999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:15.897091 2999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:43:15.898352 2999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2999
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:15.898515 2999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:15.898793 2999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:43:15.903450 2999 server_base.cc:1034] running on GCE node
W20260812 14:43:15.903405 3012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.903430 3008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:15.903405 3007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:15.903954 2999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:15.904207 2999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:15.905355 2999 hybrid_clock.cc:648] HybridClock initialized: now 1786545795905338 us; error 33 us; skew 500 ppm
I20260812 14:43:15.906662 2999 webserver.cc:460] Webserver started at http://127.15.227.195:46833/ using document root <none> and password file <none>
I20260812 14:43:15.906881 2999 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:15.906931 2999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:15.907074 2999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:15.908147 2999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "43c3a44b3a85468780cd380363c659e1"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "b81e2926fa5f97915795dba643fb4d70"
server_key_iv: "b56ceaae7dd9402f1d9ad994233d0830"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.908463 2999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "43c3a44b3a85468780cd380363c659e1"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "b81e2926fa5f97915795dba643fb4d70"
server_key_iv: "b56ceaae7dd9402f1d9ad994233d0830"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.909684 2999 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:43:15.910382 3023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.910517 2999 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.910604 2999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "43c3a44b3a85468780cd380363c659e1"
format_stamp: "Formatted at 2026-08-12 14:43:15 on dist-test-slave-q9h9"
server_key: "b81e2926fa5f97915795dba643fb4d70"
server_key_iv: "b56ceaae7dd9402f1d9ad994233d0830"
server_key_version: "encryptionkey@0"
I20260812 14:43:15.910672 2999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:15.913887 2999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:15.914151 2999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:15.914281 2999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:15.914479 2999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:15.914768 2999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:15.914819 2999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.914853 2999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:15.914880 2999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:15.920748 2999 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35633
I20260812 14:43:15.920814 3242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35633 every 8 connection(s)
I20260812 14:43:15.921100 2999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:43:15.921782 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 2999
I20260812 14:43:15.921886 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:43:15.922133 16271 external_mini_cluster.cc:1428] Setting key 9234030cd075bdbb7dbff18c69d1675a
I20260812 14:43:15.924132 2477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.921696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:45341 (local address 127.15.227.254:33337)
0812 14:43:15.921740 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.921742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.922060 (+ 318us) server_negotiation.cc:407] Connection header received
0812 14:43:15.922655 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.922657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.922664 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.922721 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.923862 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:15.923864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:15.923864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:43:15.923869 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:15.923887 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:43:15.923892 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:15.923949 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:15.924033 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:43:15.924058 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:43:15.924294 3249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:15.921695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33337 (local address 127.15.227.195:45341)
0812 14:43:15.921893 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:15.921935 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:15.922564 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:15.922789 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:15.922793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:15.922802 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:15.923711 (+ 909us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:43:15.923711 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:15.923737 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:43:15.923763 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:15.923952 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:15.923957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:15.923960 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:43:15.924040 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:43:15.924099 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260812 14:43:15.924738 3245 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33337
I20260812 14:43:15.924819 3245 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:15.924948 3245 heartbeater.cc:511] Master 127.15.227.254:33337 requested a full tablet report, sending...
I20260812 14:43:15.925287 2394 ts_manager.cc:194] Registered new tserver with Master: 43c3a44b3a85468780cd380363c659e1 (127.15.227.195:35633)
I20260812 14:43:15.933660 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 14:43:15.935357 2477 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:43:15.934947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.15.227.254:33337)
0812 14:43:15.934998 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:15.935000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:15.935073 (+ 73us) server_negotiation.cc:407] Connection header received
0812 14:43:15.935113 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:15.935114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:15.935118 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:15.935184 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:15.935305 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39196: BlockingRecv error: recv got EOF from 127.0.0.1:39196 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:43:15.935423 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 2515
I20260812 14:43:15.958915 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 2744
I20260812 14:43:15.969020 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 2999
I20260812 14:43:15.976920 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 2342
2026-08-12T14:43:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (568 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:43:16 dist-test-slave-q9h9 krb5kdc[3271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:43:16 dist-test-slave-q9h9 krb5kdc[3271](info): set up 2 sockets
Aug 12 14:43:16 dist-test-slave-q9h9 krb5kdc[3271](info): commencing operation
krb5kdc: starting...
W20260812 14:43:17.181171 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:43:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:43:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:43:17.290055 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34369
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:34369
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:43:17.369004 3298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:17.369197 3298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:17.369230 3298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:17.370388 3298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:43:17.370428 3298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:17.370448 3298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:17.370465 3298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:43:17.370488 3298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:43:17.371899 3298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:34369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:17.372123 3298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:17.372320 3298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:43:17.376791 3298 server_base.cc:1034] running on GCE node
W20260812 14:43:17.376780 3307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.376767 3308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.376776 3312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:17.377110 3298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:17.377379 3298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:17.378511 3298 hybrid_clock.cc:648] HybridClock initialized: now 1786545797378497 us; error 32 us; skew 500 ppm
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:43:17.380774 3298 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:43:17.381126 3298 webserver.cc:460] Webserver started at http://127.15.227.254:40531/ using document root <none> and password file <none>
I20260812 14:43:17.381292 3298 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:17.381330 3298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:17.381417 3298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:17.382373 3298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "094ab6efd352091548f0204241f8fae6"
server_key_iv: "3927b65083e890bc72d4a5c860c45005"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.382652 3298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "094ab6efd352091548f0204241f8fae6"
server_key_iv: "3927b65083e890bc72d4a5c860c45005"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.383773 3298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:43:17.384406 3327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.384552 3298 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:43:17.384629 3298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "094ab6efd352091548f0204241f8fae6"
server_key_iv: "3927b65083e890bc72d4a5c860c45005"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.384690 3298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:17.411369 3298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:17.412954 3298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:17.413090 3298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:17.416920 3298 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:34369
I20260812 14:43:17.416957 3407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:34369 every 8 connection(s)
I20260812 14:43:17.417273 3298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:43:17.418262 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:43:17.420437 3408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea: Bootstrap starting.
I20260812 14:43:17.420960 3408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea: Neither blocks nor log segments found. Creating new log.
I20260812 14:43:17.421209 3408 log.cc:824] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea: Log is configured to *not* fsync() on all Append() calls
I20260812 14:43:17.421840 3408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea: No bootstrap required, opened a new log
I20260812 14:43:17.423461 3408 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34369 } }
I20260812 14:43:17.423596 3408 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:43:17.423619 3408 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0224e053cf4a5c9a0df2f1aa8c07ea, State: Initialized, Role: FOLLOWER
I20260812 14:43:17.423727 3408 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34369 } }
I20260812 14:43:17.423804 3408 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:43:17.423831 3408 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:43:17.423864 3408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:43:17.424130 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 3298
I20260812 14:43:17.424244 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:43:17.424453 16271 external_mini_cluster.cc:1428] Setting key 23609cc5f978233f62da0a686bd2d0cc
I20260812 14:43:17.424836 3408 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34369 } }
I20260812 14:43:17.424944 3408 leader_election.cc:302] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd0224e053cf4a5c9a0df2f1aa8c07ea; no voters:
I20260812 14:43:17.425107 3408 leader_election.cc:288] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:43:17.425144 3417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:43:17.425340 3417 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [term 1 LEADER]: Becoming Leader. State: Replica: cd0224e053cf4a5c9a0df2f1aa8c07ea, State: Running, Role: LEADER
I20260812 14:43:17.425464 3408 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:43:17.425496 3417 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34369 } }
I20260812 14:43:17.426062 3418 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34369 } } }
I20260812 14:43:17.426064 3424 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd0224e053cf4a5c9a0df2f1aa8c07ea. Latest consensus state: current_term: 1 leader_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd0224e053cf4a5c9a0df2f1aa8c07ea" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34369 } } }
I20260812 14:43:17.426172 3418 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [sys.catalog]: This master's current role is: LEADER
I20260812 14:43:17.426185 3424 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea [sys.catalog]: This master's current role is: LEADER
I20260812 14:43:17.426391 3428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:43:17.426978 3428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:43:17.428514 3428 catalog_manager.cc:1347] Generated new cluster ID: 6b2b3e12cbe14a279a0b8a153f751b7c
I20260812 14:43:17.428562 3428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:43:17.435923 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.424930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47916 (local address 127.15.227.254:34369)
0812 14:43:17.425139 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.425150 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.425185 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:43:17.425301 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.425310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.425367 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.426332 (+ 965us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:17.426442 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.427931 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.428954 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.429582 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.429690 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.432380 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:17.432385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:17.432385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:43:17.432397 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:17.434344 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.434713 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.434716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.434717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.434861 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.435038 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.435039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.435039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.435346 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:17.435503 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.435572 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.435710 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
I20260812 14:43:17.440438 3428 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:43:17.441109 3428 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:43:17.451916 3428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea: Generated new TSK 0
I20260812 14:43:17.452064 3428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:43:17.479945 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34369
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:43:17.558440 3465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:17.558616 3465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:17.558637 3465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:17.559682 3465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:17.559713 3465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:17.559736 3465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:43:17.561081 3465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:17.561257 3465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:17.561416 3465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:17.565886 3478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.565884 3474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.565907 3475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:17.565948 3465 server_base.cc:1034] running on GCE node
I20260812 14:43:17.566334 3465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:17.566555 3465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:17.567687 3465 hybrid_clock.cc:648] HybridClock initialized: now 1786545797567674 us; error 31 us; skew 500 ppm
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:43:17.569978 3465 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:43:17.570400 3465 webserver.cc:460] Webserver started at http://127.15.227.193:36665/ using document root <none> and password file <none>
I20260812 14:43:17.570587 3465 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:17.570632 3465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:17.570765 3465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:17.571663 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "13ae3f01b26c4d28b0a22beb1422330a"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "90ddf2791429da0405a02e65cfad77b8"
server_key_iv: "e8aab9c800a170e2d91ca72c26cd1c98"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.571983 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "13ae3f01b26c4d28b0a22beb1422330a"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "90ddf2791429da0405a02e65cfad77b8"
server_key_iv: "e8aab9c800a170e2d91ca72c26cd1c98"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.573141 3465 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:43:17.573777 3490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.573933 3465 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.574030 3465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "13ae3f01b26c4d28b0a22beb1422330a"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "90ddf2791429da0405a02e65cfad77b8"
server_key_iv: "e8aab9c800a170e2d91ca72c26cd1c98"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.574100 3465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:17.583876 3465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:17.585337 3465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:17.585456 3465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:17.585644 3465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:17.585901 3465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:17.585933 3465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.585958 3465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:17.585985 3465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.591470 3465 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:38327
I20260812 14:43:17.591524 3705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:38327 every 8 connection(s)
I20260812 14:43:17.591781 3465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:43:17.593752 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 3465
I20260812 14:43:17.593849 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:43:17.594048 16271 external_mini_cluster.cc:1428] Setting key baf7d8533e03f02e2f8a044fe5875d92
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:43:17.601234 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.592367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:37969 (local address 127.15.227.254:34369)
0812 14:43:17.592413 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.592416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.592596 (+ 180us) server_negotiation.cc:407] Connection header received
0812 14:43:17.592986 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.592987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.592998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.593049 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:17.593916 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.594636 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.595906 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.596092 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.596141 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.598421 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:17.598423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:17.598423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:43:17.598432 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:17.599932 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.600388 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.600389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.600389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.600434 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.600679 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.600680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.600680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.600771 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:17.600814 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.601102 (+ 288us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.601175 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:43:17.601450 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.592334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:34369 (local address 127.15.227.193:37969)
0812 14:43:17.592484 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:17.592520 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:17.592915 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:17.593102 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:17.593105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:17.593633 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:17.593803 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.593808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.594717 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.594719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.595832 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.595835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.596169 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.596169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.596301 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.596763 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:17.596771 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:17.598268 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:17.600011 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:43:17.600019 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:43:17.600021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:43:17.600317 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:43:17.600486 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:43:17.600486 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:43:17.600487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:43:17.600616 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:43:17.600824 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:17.600829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:17.600977 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 14:43:17.601091 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 14:43:17.601251 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260812 14:43:17.601917 3706 heartbeater.cc:347] Connected to a master server at 127.15.227.254:34369
I20260812 14:43:17.602010 3706 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:17.602216 3706 heartbeater.cc:511] Master 127.15.227.254:34369 requested a full tablet report, sending...
I20260812 14:43:17.602715 3347 ts_manager.cc:194] Registered new tserver with Master: 13ae3f01b26c4d28b0a22beb1422330a (127.15.227.193:38327)
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
I20260812 14:43:17.603328 3347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:37969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:43:17.617905 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34369
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:43:17.722272 3718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:17.722445 3718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:17.722466 3718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:17.723614 3718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:17.723657 3718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:17.723685 3718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:43:17.725101 3718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:17.725309 3718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:17.725494 3718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:17.730126 3728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.730140 3731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.730175 3726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:17.730320 3718 server_base.cc:1034] running on GCE node
I20260812 14:43:17.730501 3718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:17.730691 3718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:17.731813 3718 hybrid_clock.cc:648] HybridClock initialized: now 1786545797731806 us; error 26 us; skew 500 ppm
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:43:17.734153 3718 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:43:17.734498 3718 webserver.cc:460] Webserver started at http://127.15.227.194:41783/ using document root <none> and password file <none>
I20260812 14:43:17.734671 3718 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:17.734709 3718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:17.734814 3718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:17.735754 3718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "8d0a028532eb47b1b34c7ff96ce35936"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "85be851089c0cbc53e8c441f25c0e39d"
server_key_iv: "c49be3ab82919524a782665fea882a72"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.736022 3718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "8d0a028532eb47b1b34c7ff96ce35936"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "85be851089c0cbc53e8c441f25c0e39d"
server_key_iv: "c49be3ab82919524a782665fea882a72"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.737107 3718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:43:17.737700 3746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.737844 3718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.737922 3718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "8d0a028532eb47b1b34c7ff96ce35936"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "85be851089c0cbc53e8c441f25c0e39d"
server_key_iv: "c49be3ab82919524a782665fea882a72"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.737974 3718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:17.750214 3718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:17.751720 3718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:17.751820 3718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:17.751976 3718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:17.752205 3718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:17.752234 3718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.752252 3718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:17.752269 3718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.757371 3718 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:34175
I20260812 14:43:17.757437 3966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:34175 every 8 connection(s)
I20260812 14:43:17.757683 3718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:43:17.762804 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 3718
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:43:17.762899 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:43:17.763080 16271 external_mini_cluster.cc:1428] Setting key af94af3aa3eae1ef14a66e350feac9b7
I20260812 14:43:17.765750 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.758311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:53109 (local address 127.15.227.254:34369)
0812 14:43:17.758354 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.758355 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.758511 (+ 156us) server_negotiation.cc:407] Connection header received
0812 14:43:17.758933 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.758934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.758944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.758979 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:17.759768 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.760240 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.761101 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.761283 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.761329 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.763341 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:17.763343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:17.763343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:43:17.763351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:17.764553 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.764902 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.764903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.764903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.764963 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.765167 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.765168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.765168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.765282 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:17.765337 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.765575 (+ 238us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.765661 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:43:17.765883 3970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.758263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:34369 (local address 127.15.227.194:53109)
0812 14:43:17.758412 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:17.758450 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:17.758848 (+ 398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:17.759031 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:17.759034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:17.759540 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:17.759706 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.759710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.760297 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.760298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.761042 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.761044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.761335 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.761335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.761421 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.761834 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:17.761844 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:17.763258 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:17.764625 (+ 1367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:43:17.764632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:43:17.764633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:43:17.764838 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:43:17.765014 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:43:17.765015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:43:17.765016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:43:17.765113 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:43:17.765336 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:17.765343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:17.765463 (+ 120us) client_negotiation.cc:769] Sending connection context
0812 14:43:17.765545 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:43:17.765681 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260812 14:43:17.766316 3967 heartbeater.cc:347] Connected to a master server at 127.15.227.254:34369
I20260812 14:43:17.766404 3967 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:17.766556 3967 heartbeater.cc:511] Master 127.15.227.254:34369 requested a full tablet report, sending...
I20260812 14:43:17.766903 3347 ts_manager.cc:194] Registered new tserver with Master: 8d0a028532eb47b1b34c7ff96ce35936 (127.15.227.194:34175)
I20260812 14:43:17.767254 3347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:53109
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:43:17.784052 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34369
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:37243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:43:17.861304 3974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:43:17.861477 3974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:43:17.861498 3974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:43:17.862622 3974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:43:17.862661 3974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:43:17.862690 3974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:43:17.864071 3974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:37243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34369
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:43:17.864274 3974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:43:17.864465 3974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:43:17.868877 3989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.868945 3985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:43:17.868889 3986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:43:17.869133 3974 server_base.cc:1034] running on GCE node
I20260812 14:43:17.869270 3974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:43:17.869453 3974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:43:17.870584 3974 hybrid_clock.cc:648] HybridClock initialized: now 1786545797870571 us; error 30 us; skew 500 ppm
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:43:17.872773 3974 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:43:17.873142 3974 webserver.cc:460] Webserver started at http://127.15.227.195:33029/ using document root <none> and password file <none>
I20260812 14:43:17.873340 3974 fs_manager.cc:359] Metadata directory not provided
I20260812 14:43:17.873381 3974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:43:17.873494 3974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:43:17.874492 3974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "f793959e8b064fb5911d899685072176"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "c859c1e4df933d32327452ba5459c921"
server_key_iv: "24606678dff9085f85de15894e4cdd5b"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.874881 3974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "f793959e8b064fb5911d899685072176"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "c859c1e4df933d32327452ba5459c921"
server_key_iv: "24606678dff9085f85de15894e4cdd5b"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.876055 3974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:43:17.876736 4002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.876865 3974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:43:17.876931 3974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "f793959e8b064fb5911d899685072176"
format_stamp: "Formatted at 2026-08-12 14:43:17 on dist-test-slave-q9h9"
server_key: "c859c1e4df933d32327452ba5459c921"
server_key_iv: "24606678dff9085f85de15894e4cdd5b"
server_key_version: "encryptionkey@0"
I20260812 14:43:17.876994 3974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:43:17.886725 3974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:43:17.888360 3974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:43:17.888473 3974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:43:17.888655 3974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:43:17.888917 3974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:43:17.888949 3974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.888973 3974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:43:17.889000 3974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:43:17.894218 3974 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:42341
I20260812 14:43:17.894280 4199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:42341 every 8 connection(s)
I20260812 14:43:17.894548 3974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:43:17.897877 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 3974
I20260812 14:43:17.897980 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:43:17.898185 16271 external_mini_cluster.cc:1428] Setting key e273ebcef5b91718185e78907e73e30b
Aug 12 14:43:17 dist-test-slave-q9h9 krb5kdc[3271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545797, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:43:17.902729 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.895160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:40525 (local address 127.15.227.254:34369)
0812 14:43:17.895228 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.895231 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.895350 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:43:17.895752 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.895753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.895764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.895801 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:17.896581 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.897284 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.898549 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.898827 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.898883 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.900705 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:17.900707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:17.900707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:43:17.900713 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:17.901707 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.902039 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.902040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.902040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.902096 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.902289 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.902289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.902289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.902378 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:17.902421 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.902615 (+ 194us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.902669 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:43:17.902873 4205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.895107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:34369 (local address 127.15.227.195:40525)
0812 14:43:17.895250 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:17.895281 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:17.895669 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:17.895846 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:17.895849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:17.896343 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:43:17.896515 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.896519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.897383 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.897385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.898434 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.898437 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.898901 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.898902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.899030 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.899405 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:43:17.899412 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:43:17.900629 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:43:17.901767 (+ 1138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:43:17.901773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:43:17.901774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:43:17.901963 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:43:17.902150 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:43:17.902151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:43:17.902151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:43:17.902241 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:43:17.902422 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:43:17.902424 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:43:17.902519 (+ 95us) client_negotiation.cc:769] Sending connection context
0812 14:43:17.902596 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:43:17.902701 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260812 14:43:17.903270 4200 heartbeater.cc:347] Connected to a master server at 127.15.227.254:34369
I20260812 14:43:17.903343 4200 heartbeater.cc:464] Registering TS with master...
I20260812 14:43:17.903467 4200 heartbeater.cc:511] Master 127.15.227.254:34369 requested a full tablet report, sending...
I20260812 14:43:17.903776 3347 ts_manager.cc:194] Registered new tserver with Master: f793959e8b064fb5911d899685072176 (127.15.227.195:42341)
I20260812 14:43:17.904098 3347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:40525
I20260812 14:43:17.909626 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:43:17.915771 3416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.910813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33636 (local address 127.15.227.254:34369)
0812 14:43:17.910855 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.910857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.910957 (+ 100us) server_negotiation.cc:407] Connection header received
0812 14:43:17.910993 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.910994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.911001 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.911043 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:43:17.911546 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.911978 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.912616 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.912796 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.912846 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.913783 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:43:17.913784 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:43:17.913784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:43:17.913789 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:43:17.915020 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.915266 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.915266 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.915266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.915307 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:43:17.915416 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:43:17.915417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:43:17.915417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:43:17.915529 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:43:17.915597 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.915664 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.915715 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:43:17.917028 3347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:43:17.918622 3347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:43:17.924417 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.920446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:42341 (local address 127.0.0.1:55550)
0812 14:43:17.920695 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:17.920734 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:17.920795 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:17.920989 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:17.920991 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:17.920993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:43:17.921051 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.921053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.922150 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.922151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.923376 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.923379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.924164 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.924165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.924240 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.924254 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:43:17.924314 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:43:17.924362 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":49,"threads_started":1}
I20260812 14:43:17.924649 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.920447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34175 (local address 127.0.0.1:38418)
0812 14:43:17.920730 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:17.920741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:17.920799 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:17.921081 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:17.921082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:17.921084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:43:17.921137 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.921139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.922448 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.922449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.923510 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.923512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.924410 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.924411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.924476 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.924479 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:43:17.924525 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:43:17.924560 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":1,"thread_start_us":49,"threads_started":1}
I20260812 14:43:17.924686 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.920586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.15.227.195:42341)
0812 14:43:17.920793 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.920798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.920808 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:43:17.920898 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.920901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.920916 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.920984 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:43:17.921112 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.922067 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.923443 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.924085 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.924156 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.924204 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.924575 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.924614 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260812 14:43:17.924947 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.920614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.15.227.194:34175)
0812 14:43:17.920880 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.920884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.920893 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:43:17.920931 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.920933 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.920948 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.921014 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:43:17.921210 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.922352 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.923585 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.924267 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.924326 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.924376 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.924824 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.924867 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":74,"threads_started":1}
I20260812 14:43:17.925241 4082 tablet_service.cc:1467] Processing CreateTablet for tablet 77498104f7b044c1aeee86df8ac3830e (DEFAULT_TABLE table=test-table [id=2f71d217a2b64b1895b10a6fbeaeb9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:43:17.925482 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.920465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:38327 (local address 127.0.0.1:52786)
0812 14:43:17.920886 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:17.920894 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:17.920940 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:17.921156 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:17.921157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:17.921159 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:43:17.921197 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.921198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.923574 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.923574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.924236 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:17.924237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.925205 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.925206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:17.925321 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.925326 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:43:17.925378 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:43:17.925418 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":80,"threads_started":1}
I20260812 14:43:17.925563 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:17.920670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.15.227.193:38327)
0812 14:43:17.920895 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:17.920899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:17.920936 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:43:17.921042 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:17.921045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:17.921060 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:17.921153 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:43:17.921249 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.922343 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.924328 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:17.925034 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:17.925114 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:17.925169 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:17.925445 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 14:43:17.925488 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260812 14:43:17.925571 4082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77498104f7b044c1aeee86df8ac3830e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:43:17.925560 3835 tablet_service.cc:1467] Processing CreateTablet for tablet 77498104f7b044c1aeee86df8ac3830e (DEFAULT_TABLE table=test-table [id=2f71d217a2b64b1895b10a6fbeaeb9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:43:17.925879 3835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77498104f7b044c1aeee86df8ac3830e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:43:17.926518 3577 tablet_service.cc:1467] Processing CreateTablet for tablet 77498104f7b044c1aeee86df8ac3830e (DEFAULT_TABLE table=test-table [id=2f71d217a2b64b1895b10a6fbeaeb9e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:43:17.926832 3577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77498104f7b044c1aeee86df8ac3830e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:43:17.928514 4227 tablet_bootstrap.cc:492] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Bootstrap starting.
I20260812 14:43:17.929122 4227 tablet_bootstrap.cc:654] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Neither blocks nor log segments found. Creating new log.
I20260812 14:43:17.929576 4227 log.cc:824] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Log is configured to *not* fsync() on all Append() calls
I20260812 14:43:17.929759 4229 tablet_bootstrap.cc:492] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a: Bootstrap starting.
I20260812 14:43:17.930130 4228 tablet_bootstrap.cc:492] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Bootstrap starting.
I20260812 14:43:17.930419 4229 tablet_bootstrap.cc:654] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a: Neither blocks nor log segments found. Creating new log.
I20260812 14:43:17.930459 4227 tablet_bootstrap.cc:492] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: No bootstrap required, opened a new log
I20260812 14:43:17.930517 4227 ts_tablet_manager.cc:1397] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:43:17.930773 4228 tablet_bootstrap.cc:654] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Neither blocks nor log segments found. Creating new log.
I20260812 14:43:17.931008 4229 log.cc:824] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:43:17.931296 4228 log.cc:824] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Log is configured to *not* fsync() on all Append() calls
I20260812 14:43:17.931777 4229 tablet_bootstrap.cc:492] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a: No bootstrap required, opened a new log
I20260812 14:43:17.931879 4229 ts_tablet_manager.cc:1397] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:43:17.931913 4228 tablet_bootstrap.cc:492] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: No bootstrap required, opened a new log
I20260812 14:43:17.931964 4228 ts_tablet_manager.cc:1397] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:43:17.932183 4227 raft_consensus.cc:348] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:17.932310 4227 raft_consensus.cc:374] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:43:17.932343 4227 raft_consensus.cc:729] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d0a028532eb47b1b34c7ff96ce35936, State: Initialized, Role: FOLLOWER
I20260812 14:43:17.932471 4227 consensus_queue.cc:260] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:17.932735 4227 ts_tablet_manager.cc:1428] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:43:17.932735 3967 heartbeater.cc:503] Master 127.15.227.254:34369 was elected leader, sending a full tablet report...
I20260812 14:43:17.933737 4229 raft_consensus.cc:348] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:17.933730 4228 raft_consensus.cc:348] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:17.933885 4228 raft_consensus.cc:374] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:43:17.933884 4229 raft_consensus.cc:374] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:43:17.933930 4228 raft_consensus.cc:729] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f793959e8b064fb5911d899685072176, State: Initialized, Role: FOLLOWER
I20260812 14:43:17.933930 4229 raft_consensus.cc:729] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ae3f01b26c4d28b0a22beb1422330a, State: Initialized, Role: FOLLOWER
I20260812 14:43:17.934073 4229 consensus_queue.cc:260] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:17.934079 4228 consensus_queue.cc:260] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:17.934343 4229 ts_tablet_manager.cc:1428] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:43:17.934343 4228 ts_tablet_manager.cc:1428] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:43:17.934352 4200 heartbeater.cc:503] Master 127.15.227.254:34369 was elected leader, sending a full tablet report...
I20260812 14:43:17.934422 3706 heartbeater.cc:503] Master 127.15.227.254:34369 was elected leader, sending a full tablet report...
W20260812 14:43:18.008174 3968 tablet.cc:2369] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:43:18.041503 4242 raft_consensus.cc:482] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:43:18.041644 4242 raft_consensus.cc:504] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:18.042064 4242 leader_election.cc:288] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d0a028532eb47b1b34c7ff96ce35936 (127.15.227.194:34175), f793959e8b064fb5911d899685072176 (127.15.227.195:42341)
I20260812 14:43:18.044687 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:18.042259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:56441 (local address 127.15.227.194:34175)
0812 14:43:18.042326 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:18.042329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:18.042340 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:43:18.042369 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:18.042370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:18.042381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:18.042440 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:43:18.042610 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.043075 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.044008 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.044375 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.044438 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:18.044447 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:18.044587 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:43:18.044629 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:43:18.044687 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:18.042191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34175 (local address 127.15.227.193:56441)
0812 14:43:18.042257 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:18.042269 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:18.042310 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:18.042465 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:18.042466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:18.042468 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:43:18.042535 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:18.042537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.043155 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.043156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:18.043919 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:18.043921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.044451 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.044452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:18.044522 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:18.044525 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:43:18.044572 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:43:18.044613 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":24}
I20260812 14:43:18.044766 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:18.042371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:38193 (local address 127.15.227.195:42341)
0812 14:43:18.042424 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:18.042426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:18.042514 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:43:18.042559 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:18.042560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:18.042569 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:18.042613 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:43:18.042757 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.043202 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.044092 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.044460 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.044519 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:18.044527 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:18.044659 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:43:18.044694 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:43:18.044801 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:18.042266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:42341 (local address 127.15.227.193:38193)
0812 14:43:18.042452 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:18.042460 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:43:18.042506 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:43:18.042622 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:43:18.042622 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:43:18.042624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:43:18.042696 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:18.042698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.043290 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.043291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:18.044010 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:43:18.044011 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.044527 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.044528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:43:18.044598 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:18.044602 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:43:18.044646 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:43:18.044683 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":123,"threads_started":1}
I20260812 14:43:18.045053 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77498104f7b044c1aeee86df8ac3830e" candidate_uuid: "13ae3f01b26c4d28b0a22beb1422330a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d0a028532eb47b1b34c7ff96ce35936" is_pre_election: true
I20260812 14:43:18.045202 3882 raft_consensus.cc:2454] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ae3f01b26c4d28b0a22beb1422330a in term 0.
I20260812 14:43:18.045145 4114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77498104f7b044c1aeee86df8ac3830e" candidate_uuid: "13ae3f01b26c4d28b0a22beb1422330a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f793959e8b064fb5911d899685072176" is_pre_election: true
I20260812 14:43:18.045303 4114 raft_consensus.cc:2454] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13ae3f01b26c4d28b0a22beb1422330a in term 0.
I20260812 14:43:18.045423 3495 leader_election.cc:302] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ae3f01b26c4d28b0a22beb1422330a, 8d0a028532eb47b1b34c7ff96ce35936; no voters:
I20260812 14:43:18.045579 4242 raft_consensus.cc:2781] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:43:18.045624 4242 raft_consensus.cc:482] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:43:18.045639 4242 raft_consensus.cc:3037] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:43:18.046424 4242 raft_consensus.cc:504] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:18.046550 4242 leader_election.cc:288] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [CANDIDATE]: Term 1 election: Requested vote from peers 8d0a028532eb47b1b34c7ff96ce35936 (127.15.227.194:34175), f793959e8b064fb5911d899685072176 (127.15.227.195:42341)
I20260812 14:43:18.046669 3882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77498104f7b044c1aeee86df8ac3830e" candidate_uuid: "13ae3f01b26c4d28b0a22beb1422330a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d0a028532eb47b1b34c7ff96ce35936"
I20260812 14:43:18.046681 4114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77498104f7b044c1aeee86df8ac3830e" candidate_uuid: "13ae3f01b26c4d28b0a22beb1422330a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f793959e8b064fb5911d899685072176"
I20260812 14:43:18.046746 4114 raft_consensus.cc:3037] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:43:18.046747 3882 raft_consensus.cc:3037] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:43:18.047433 4114 raft_consensus.cc:2454] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ae3f01b26c4d28b0a22beb1422330a in term 1.
I20260812 14:43:18.047502 3882 raft_consensus.cc:2454] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13ae3f01b26c4d28b0a22beb1422330a in term 1.
I20260812 14:43:18.047641 3497 leader_election.cc:302] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ae3f01b26c4d28b0a22beb1422330a, f793959e8b064fb5911d899685072176; no voters:
I20260812 14:43:18.047740 4242 raft_consensus.cc:2781] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:43:18.047868 4242 raft_consensus.cc:686] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [term 1 LEADER]: Becoming Leader. State: Replica: 13ae3f01b26c4d28b0a22beb1422330a, State: Running, Role: LEADER
I20260812 14:43:18.047947 4242 consensus_queue.cc:237] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } }
I20260812 14:43:18.048728 3347 catalog_manager.cc:5495] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a reported cstate change: term changed from 0 to 1, leader changed from <none> to 13ae3f01b26c4d28b0a22beb1422330a (127.15.227.193). New cstate: current_term: 1 leader_uuid: "13ae3f01b26c4d28b0a22beb1422330a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13ae3f01b26c4d28b0a22beb1422330a" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 38327 } health_report { overall_health: HEALTHY } } }
I20260812 14:43:18.069711 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:18.067538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.15.227.193:38327)
0812 14:43:18.067593 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:43:18.067595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:43:18.067608 (+ 13us) server_negotiation.cc:407] Connection header received
0812 14:43:18.067675 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:43:18.067676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:43:18.067694 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:43:18.067756 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:43:18.067857 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.068307 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.068985 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:43:18.069242 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:43:18.069296 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:43:18.069442 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:43:18.069533 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:43:18.069597 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:43:18.069613 (+ 16us) server_negotiation.cc:299] Negotiation successful
0812 14:43:18.069630 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:43:18.071321 4114 raft_consensus.cc:1261] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 1 FOLLOWER]: Refusing update from remote peer 13ae3f01b26c4d28b0a22beb1422330a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:43:18.071327 3882 raft_consensus.cc:1261] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 1 FOLLOWER]: Refusing update from remote peer 13ae3f01b26c4d28b0a22beb1422330a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:43:18.071655 4244 consensus_queue.cc:1035] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d0a028532eb47b1b34c7ff96ce35936" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:43:18.071787 4242 consensus_queue.cc:1035] T 77498104f7b044c1aeee86df8ac3830e P 13ae3f01b26c4d28b0a22beb1422330a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f793959e8b064fb5911d899685072176" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:43:18.074366 4248 mvcc.cc:205] Tried to move back new op lower bound from 7317691588898021376 to 7317691588804980736. Current Snapshot: MvccSnapshot[applied={T|T < 7317691588898021376}]
I20260812 14:43:18.074777 4249 mvcc.cc:205] Tried to move back new op lower bound from 7317691588898021376 to 7317691588804980736. Current Snapshot: MvccSnapshot[applied={T|T < 7317691588898021376}]
I20260812 14:43:18.075168 4252 mvcc.cc:205] Tried to move back new op lower bound from 7317691588898021376 to 7317691588804980736. Current Snapshot: MvccSnapshot[applied={T|T < 7317691588898021376}]
I20260812 14:43:18.076802 3347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:43:18.076902 3347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:43:18.077714 3347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd0224e053cf4a5c9a0df2f1aa8c07ea: Sending DeleteTablet for 3 replicas of tablet 77498104f7b044c1aeee86df8ac3830e
I20260812 14:43:18.078099 4082 tablet_service.cc:1514] Processing DeleteTablet for tablet 77498104f7b044c1aeee86df8ac3830e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:43:18 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55550
I20260812 14:43:18.078101 3577 tablet_service.cc:1514] Processing DeleteTablet for tablet 77498104f7b044c1aeee86df8ac3830e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:43:18 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:52786
I20260812 14:43:18.079581 3835 tablet_service.cc:1514] Processing DeleteTablet for tablet 77498104f7b044c1aeee86df8ac3830e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:43:18 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:38418
I20260812 14:43:18.079793 4264 tablet_replica.cc:330] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: stopping tablet replica
I20260812 14:43:18.079864 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 3465
I20260812 14:43:18.079873 4262 tablet_replica.cc:330] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: stopping tablet replica
I20260812 14:43:18.079926 4264 raft_consensus.cc:2229] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:43:18.079954 4262 raft_consensus.cc:2229] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:43:18.080032 4262 raft_consensus.cc:2258] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:43:18.080025 4264 raft_consensus.cc:2258] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:43:18.083158 4264 ts_tablet_manager.cc:1905] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:43:18.083554 4262 ts_tablet_manager.cc:1905] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:43:18.084748 4264 ts_tablet_manager.cc:1918] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:43:18.084833 4264 log.cc:1196] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/77498104f7b044c1aeee86df8ac3830e
I20260812 14:43:18.085126 4262 ts_tablet_manager.cc:1918] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:43:18.085184 4264 ts_tablet_manager.cc:1939] T 77498104f7b044c1aeee86df8ac3830e P 8d0a028532eb47b1b34c7ff96ce35936: Deleting consensus metadata
I20260812 14:43:18.085207 4262 log.cc:1196] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/77498104f7b044c1aeee86df8ac3830e
I20260812 14:43:18.085559 4262 ts_tablet_manager.cc:1939] T 77498104f7b044c1aeee86df8ac3830e P f793959e8b064fb5911d899685072176: Deleting consensus metadata
I20260812 14:43:18.085609 3330 catalog_manager.cc:4841] TS 8d0a028532eb47b1b34c7ff96ce35936 (127.15.227.194:34175): tablet 77498104f7b044c1aeee86df8ac3830e (table test-table [id=2f71d217a2b64b1895b10a6fbeaeb9e4]) successfully deleted
I20260812 14:43:18.085980 3331 catalog_manager.cc:4841] TS f793959e8b064fb5911d899685072176 (127.15.227.195:42341): tablet 77498104f7b044c1aeee86df8ac3830e (table test-table [id=2f71d217a2b64b1895b10a6fbeaeb9e4]) successfully deleted
W20260812 14:43:18.088668 3329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:38327 (error 108)
I20260812 14:43:18.088883 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 3718
I20260812 14:43:18.089143 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:43:18.089034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:38327 (local address 127.0.0.1:52818)
0812 14:43:18.089080 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:43:18.089113 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:43:18.089321 3329 catalog_manager.cc:4584] TS 13ae3f01b26c4d28b0a22beb1422330a (127.15.227.193:38327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77498104f7b044c1aeee86df8ac3830e: Network error: Client connection negotiation failed: client connection to 127.15.227.193:38327: connect: Connection refused (error 111)
I20260812 14:43:18.100190 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 3974
I20260812 14:43:18.114355 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 3298
2026-08-12T14:43:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2139 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:43:18 dist-test-slave-q9h9 krb5kdc[4277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:43:18 dist-test-slave-q9h9 krb5kdc[4277](info): set up 2 sockets
Aug 12 14:43:18 dist-test-slave-q9h9 krb5kdc[4277](info): commencing operation
krb5kdc: starting...
W20260812 14:43:19.320070 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:43:19 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:43:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/rangeradmin_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/rangeradmin_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/rangerlookup_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/rangerlookup_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
I20260812 14:43:19.460366 16271 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskN3u55Q/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/postgres -l logfile start
2026-08-12 14:43:21.542 UTC [4338] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 14:43:21.542 UTC [4338] LOG: listening on IPv4 address "127.15.227.212", port 37067
2026-08-12 14:43:21.545 UTC [4338] LOG: listening on Unix socket "/tmp/.s.PGSQL.37067"
2026-08-12 14:43:21.563 UTC [4344] LOG: database system was shut down at 2026-08-12 14:43:20 UTC
2026-08-12 14:43:21.566 UTC [4338] LOG: database system is ready to accept connections
I20260812 14:43:22.678099 16271 mini_postgres.cc:96] Postgres bound to 37067
2026-08-12 14:43:22.686 UTC [4357] FATAL: database "slave" does not exist
127.15.227.212:37067 - accepting connections
I20260812 14:43:22.686949 16271 mini_ranger.cc:161] Starting Ranger...
I20260812 14:43:22.701885 16271 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 14:43:23.058459 16271 mini_ranger.cc:87] Created ranger Postgres database
I20260812 14:43:23.058545 16271 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-admin
2026-08-12 14:43:23,379 [I] DB FLAVOR :POSTGRES
2026-08-12 14:43:23,380 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:43:23,380 [I] Checking connection..
2026-08-12 14:43:23,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 14:43:23,643 [I] Checking connection passed.
2026-08-12 14:43:23,643 [I] --------- Verifying version history table ---------
2026-08-12 14:43:23,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:43:23,866 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:43:23,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:43:24,119 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:43:24,119 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 14:43:24,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 14:43:24.348 UTC [4471] WARNING: there is no transaction in progress
2026-08-12 14:43:24,358 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 14:43:24,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:43:24,607 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 14:43:24,608 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:43:24,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 14:43:24,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 14:43:25,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-q9h9.c.gcp-upstream.internal','N') ;"
2026-08-12 14:43:25,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:43:25,575 [I] Table x_portal_user does not exist in database ranger
2026-08-12 14:43:25,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:43:25,800 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 14:43:25,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:43:26,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:43:26,280 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 14:43:26,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 14:43:27.129 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.143 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.157 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.170 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.186 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.200 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.249 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.255 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.267 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.276 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.285 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.297 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.305 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.310 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.316 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27.321 UTC [4684] WARNING: there is no transaction in progress
2026-08-12 14:43:27,828 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 14:43:27,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 14:43:28,068 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 14:43:28,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:43:28,325 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 14:43:28,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:43:28,566 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 14:43:28,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:43:28,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:43:29,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-q9h9.c.gcp-upstream.internal';"
2026-08-12 14:43:29,288 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 14:43:29,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 14:43:29,510 [I] DB_PATCHES have already been applied
I20260812 14:43:29.516043 16271 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-admin:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskN3u55Q/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 14:43:29.516103 16271 mini_ranger.cc:193] Using host: 127.15.227.212
I20260812 14:43:29.519734 16271 mini_ranger.cc:227] Ranger admin URL: http://127.15.227.212:40211
Aug 12, 2026 2:43:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:43:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:43:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:43:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:43:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:43:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:43:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 14:43:30.849175 16271 mini_ranger.cc:160] Time spent starting Ranger: real 8.162s user 0.000s sys 0.010s
Aug 12, 2026 2:43:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:43:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 2:43:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:43:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:43:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 14:43:52.058503 16271 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/rangerkms_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/rangerkms_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.227.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.227.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 14:43:52 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545832, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 14:43:52.153446 16271 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 14:43:52.163841 16271 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 14:43:52.813692 16271 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 14:43:52.813782 16271 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms
I20260812 14:43:52.813802 16271 mini_ranger_kms.cc:227] Using postgres at 127.15.227.212:37067
2026-08-12 14:43:53,054 [I] DB FLAVOR :POSTGRES
2026-08-12 14:43:53,054 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:43:53,055 [I] Checking connection
2026-08-12 14:43:53,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 14:43:53,289 [I] connection success
2026-08-12 14:43:53,289 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 14:43:53,289 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 14:43:53,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 14:43:53,521 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 14:43:53,521 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:43:53,521 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 14:43:53,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:37067/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 14:43:53,769 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 14:43:54.211131 16271 mini_ranger_kms.cc:310] Created kms service
I20260812 14:43:54.374320 16271 mini_ranger_kms.cc:326] Created kudu user
I20260812 14:43:54.413842 16271 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 14:43:54 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545832, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:43:54.825862 16271 mini_ranger_kms.cc:384] Added ranger policy
I20260812 14:43:54.825979 16271 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms:/tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 14:43:54.825999 16271 mini_ranger_kms.cc:240] Using host: 127.15.227.212
I20260812 14:43:54.829990 16271 mini_ranger_kms.cc:276] Ranger KMS PID: 5031
I20260812 14:43:54.830152 16271 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.227.212:57931
Aug 12, 2026 2:43:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:43:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:43:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:43:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:43:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:43:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 14:43:56 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786545836, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.227.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:43:57.430737 16271 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.277s user 0.007s sys 0.196s
I20260812 14:43:57.430850 16271 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 14:43:57.430931 16271 mini_ranger_kms.cc:401] 127.15.227.212:57931/kms/v1/keys
Aug 12 14:43:57 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545836, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.227.212@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
Aug 12 14:43:57 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545836, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.227.212@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
Aug 12 14:44:00 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:44:00.085984 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:40137
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:40137
--ranger_config_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:57931/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:00.170538 5125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:00.170746 5125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:00.170778 5125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:00.171919 5125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:00.171960 5125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:00.171979 5125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:00.171995 5125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:00.172011 5125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:00.173667 5125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.227.212:57931/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:40137
--ranger_config_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:40137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:00.173913 5125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:00.174153 5125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:00.179183 5136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:00.179352 5135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:00.179195 5141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:00.181377 5125 server_base.cc:1034] running on GCE node
I20260812 14:44:00.181639 5125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:00.181917 5125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:00.183056 5125 hybrid_clock.cc:648] HybridClock initialized: now 1786545840183040 us; error 28 us; skew 500 ppm
Aug 12 14:44:00 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:00.189903 5125 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:00.190392 5125 webserver.cc:460] Webserver started at http://127.15.227.254:39347/ using document root <none> and password file <none>
I20260812 14:44:00.190594 5125 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:00.190645 5125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:00.190774 5125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:00 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545840, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:00.241871 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "8daff9ac812a40f983269868f660b936"
format_stamp: "Formatted at 2026-08-12 14:44:00 on dist-test-slave-q9h9"
server_key: "0415a4be80c6ba662c719721c05c17ba"
server_key_iv: "82beedee4cb42d89f1a12112a8e215f7"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:00.242375 5125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "8daff9ac812a40f983269868f660b936"
format_stamp: "Formatted at 2026-08-12 14:44:00 on dist-test-slave-q9h9"
server_key: "0415a4be80c6ba662c719721c05c17ba"
server_key_iv: "82beedee4cb42d89f1a12112a8e215f7"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:00.244127 5125 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:00.255789 5152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:00.255993 5125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:44:00.256111 5125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "8daff9ac812a40f983269868f660b936"
format_stamp: "Formatted at 2026-08-12 14:44:00 on dist-test-slave-q9h9"
server_key: "0415a4be80c6ba662c719721c05c17ba"
server_key_iv: "82beedee4cb42d89f1a12112a8e215f7"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:00.256208 5125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:00.264523 5125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:00.266335 5125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:00.266471 5125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:00.270680 5125 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:40137
I20260812 14:44:00.270749 5259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:40137 every 8 connection(s)
I20260812 14:44:00.271140 5125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:00.271831 5260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:00.272377 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 5125
I20260812 14:44:00.272485 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:44:00.272672 16271 external_mini_cluster.cc:1428] Setting key 2e3f8e94aaec904c065bbd0bea763d90
I20260812 14:44:00.275010 5260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936: Bootstrap starting.
I20260812 14:44:00.275782 5260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:00.276131 5260 log.cc:824] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:00.277940 5260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936: No bootstrap required, opened a new log
Aug 12 14:44:00 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:00.280503 5260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daff9ac812a40f983269868f660b936" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40137 } }
I20260812 14:44:00.280702 5260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:00.280781 5260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8daff9ac812a40f983269868f660b936, State: Initialized, Role: FOLLOWER
I20260812 14:44:00.280948 5260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daff9ac812a40f983269868f660b936" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40137 } }
I20260812 14:44:00.281037 5260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:44:00.281067 5260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:44:00.281133 5260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:00.281800 5260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daff9ac812a40f983269868f660b936" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40137 } }
I20260812 14:44:00.281908 5260 leader_election.cc:302] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8daff9ac812a40f983269868f660b936; no voters:
I20260812 14:44:00.282099 5260 leader_election.cc:288] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:44:00.282402 5260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:00.282135 5270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:00.282761 5266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:00.273212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40138 (local address 127.15.227.254:40137)
0812 14:44:00.273409 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:00.273418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:00.273510 (+ 92us) server_negotiation.cc:407] Connection header received
0812 14:44:00.274068 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:00.274076 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:00.274127 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:00.274253 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:00.274871 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:00.275768 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:00.276749 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:00.277379 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:00.277492 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:00.279679 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:00.279683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:00.279683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:00.279693 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:00.281280 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:00.281591 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:00.281594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:00.281594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:00.281676 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:00.281846 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:00.281847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:00.281847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:00.282175 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:00.282324 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:00.282428 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:44:00.282536 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260812 14:44:00.282778 5270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [term 1 LEADER]: Becoming Leader. State: Replica: 8daff9ac812a40f983269868f660b936, State: Running, Role: LEADER
I20260812 14:44:00.283113 5270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daff9ac812a40f983269868f660b936" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40137 } }
I20260812 14:44:00.283746 5270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8daff9ac812a40f983269868f660b936. Latest consensus state: current_term: 1 leader_uuid: "8daff9ac812a40f983269868f660b936" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daff9ac812a40f983269868f660b936" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40137 } } }
I20260812 14:44:00.283828 5270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:00.283753 5274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8daff9ac812a40f983269868f660b936" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daff9ac812a40f983269868f660b936" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40137 } } }
I20260812 14:44:00.284058 5274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:00.284657 5280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:00.285137 5280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:44:00.285127 5260 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:44:00.286680 5280 catalog_manager.cc:1347] Generated new cluster ID: d83be73d266d42d19aa94c6007b383b0
I20260812 14:44:00.286733 5280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:44:00.300072 5280 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:44:00.300630 5280 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:44:00.311280 5280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936: Generated new TSK 0
I20260812 14:44:00.311512 5280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:44:00.957983 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:57931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:40137
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:01.059862 5354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:01.060104 5354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:01.060137 5354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:01.061797 5354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:01.061849 5354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:01.061885 5354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:44:01.063400 5354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.227.212:57931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:40137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:01.063599 5354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:01.063808 5354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:01.068511 5365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:01.068521 5368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:01.068749 5366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:01.068887 5354 server_base.cc:1034] running on GCE node
I20260812 14:44:01.069088 5354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:01.069348 5354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:01.070521 5354 hybrid_clock.cc:648] HybridClock initialized: now 1786545841070501 us; error 33 us; skew 500 ppm
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:01.073486 5354 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:44:01.074039 5354 webserver.cc:460] Webserver started at http://127.15.227.193:32961/ using document root <none> and password file <none>
I20260812 14:44:01.074322 5354 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:01.074411 5354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:01.074551 5354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:01.087996 5354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "2ceb2ac8c05e437dad8947ba791d6c71"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "6343f3e40ee9181d37b0f286a3967d10"
server_key_iv: "b51f2eb2cee1df2e60244b7e82afbd94"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.088429 5354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "2ceb2ac8c05e437dad8947ba791d6c71"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "6343f3e40ee9181d37b0f286a3967d10"
server_key_iv: "b51f2eb2cee1df2e60244b7e82afbd94"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.090303 5354 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 14:44:01.103224 5382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.103430 5354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:01.103521 5354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "2ceb2ac8c05e437dad8947ba791d6c71"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "6343f3e40ee9181d37b0f286a3967d10"
server_key_iv: "b51f2eb2cee1df2e60244b7e82afbd94"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.103627 5354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:01.130398 5354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:01.132200 5354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:01.132335 5354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:01.132576 5354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:01.133005 5354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:01.133085 5354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.133149 5354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:01.133186 5354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.140766 5354 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:36461
I20260812 14:44:01.141228 5354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:44:01.143795 5598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:36461 every 8 connection(s)
I20260812 14:44:01.146457 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 5354
I20260812 14:44:01.146576 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:44:01.146766 16271 external_mini_cluster.cc:1428] Setting key 4969d9ce24c332371d9ad8ac89bc573a
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:01.152333 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.142006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:43295 (local address 127.15.227.254:40137)
0812 14:44:01.143976 (+ 1970us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.143978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.143988 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:44:01.144016 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.144016 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.144029 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.144096 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:01.144653 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.145163 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.146166 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.146443 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.146514 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.148902 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:01.148903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:01.148904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:01.148912 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:01.150523 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.151001 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.151002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.151002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.151071 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.151540 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.151541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.151541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.151710 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:01.151797 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.152132 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.152227 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1936,"thread_start_us":96,"threads_started":1}
I20260812 14:44:01.152624 5603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.141959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40137 (local address 127.15.227.193:43295)
0812 14:44:01.143144 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.143180 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.143636 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.144195 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.144199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.144376 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:01.144560 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.144564 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.145244 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.145246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.146088 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.146091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.146542 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.146544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.146714 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.147188 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:01.147198 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:01.148797 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:01.150615 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:01.150623 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:01.150625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:01.150920 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:01.151278 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:01.151279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:01.151280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:01.151429 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:01.151799 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:01.151806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:01.151972 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.152098 (+ 126us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.152260 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1140,"thread_start_us":109,"threads_started":1}
I20260812 14:44:01.153350 5600 heartbeater.cc:347] Connected to a master server at 127.15.227.254:40137
I20260812 14:44:01.153475 5600 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:01.153684 5600 heartbeater.cc:511] Master 127.15.227.254:40137 requested a full tablet report, sending...
I20260812 14:44:01.154311 5188 ts_manager.cc:194] Registered new tserver with Master: 2ceb2ac8c05e437dad8947ba791d6c71 (127.15.227.193:36461)
I20260812 14:44:01.155249 5188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:43295
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:44:01.181771 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:57931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:40137
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 14:44:01.298688 5611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:01.299038 5611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:01.299124 5611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:01.300809 5611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:01.300916 5611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:01.300994 5611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:44:01.303179 5611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.227.212:57931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:40137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:01.303622 5611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:01.303921 5611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:01.309983 5627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:01.310098 5624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:01.310189 5620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:01.311245 5611 server_base.cc:1034] running on GCE node
I20260812 14:44:01.311441 5611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:01.311677 5611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:01.314947 5611 hybrid_clock.cc:648] HybridClock initialized: now 1786545841314919 us; error 39 us; skew 500 ppm
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:01.318094 5611 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:44:01.318526 5611 webserver.cc:460] Webserver started at http://127.15.227.194:34197/ using document root <none> and password file <none>
I20260812 14:44:01.318749 5611 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:01.318794 5611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:01.318903 5611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:01.331362 5611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "9662769b99cc4b9fadfa161a9471d0c0"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "957c27b3e1cd7bb8ba649d0994dee53c"
server_key_iv: "210844e9ed7c002d550dc6e4547a9156"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.331822 5611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "9662769b99cc4b9fadfa161a9471d0c0"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "957c27b3e1cd7bb8ba649d0994dee53c"
server_key_iv: "210844e9ed7c002d550dc6e4547a9156"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.333411 5611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:01.349470 5644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.349686 5611 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260812 14:44:01.349783 5611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "9662769b99cc4b9fadfa161a9471d0c0"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "957c27b3e1cd7bb8ba649d0994dee53c"
server_key_iv: "210844e9ed7c002d550dc6e4547a9156"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.349860 5611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:01.373435 5611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:01.375190 5611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:01.375313 5611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:01.375521 5611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:01.375847 5611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:01.375882 5611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.375910 5611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:01.375962 5611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.382728 5611 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:35483
I20260812 14:44:01.382769 5850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:35483 every 8 connection(s)
I20260812 14:44:01.383167 5611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:44:01.388862 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 5611
I20260812 14:44:01.388980 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:44:01.389159 16271 external_mini_cluster.cc:1428] Setting key bf560d99cbe75192904eb723bef4cf16
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:01.394353 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.384705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:52481 (local address 127.15.227.254:40137)
0812 14:44:01.384776 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.384778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.384995 (+ 217us) server_negotiation.cc:407] Connection header received
0812 14:44:01.385630 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.385631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.385641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.385681 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:01.386381 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.387133 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.388570 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.388805 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.388868 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.391132 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:01.391133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:01.391134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:01.391141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:01.392746 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.393228 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.393230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.393230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.393297 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.393568 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.393569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.393569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.393712 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:01.393787 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.394152 (+ 365us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.394253 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:44:01.394847 5855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.384662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40137 (local address 127.15.227.194:52481)
0812 14:44:01.384867 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.384917 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.385531 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.385749 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.385754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.385977 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:01.386273 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.386281 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.387241 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.387242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.388494 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.388497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.388893 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.388894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.389031 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.389476 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:01.389486 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:01.391044 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:01.392832 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:01.392838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:01.392840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:01.393092 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:01.393364 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:01.393364 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:01.393365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:01.393494 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:01.393782 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:01.393787 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:01.393956 (+ 169us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.394474 (+ 518us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.394624 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260812 14:44:01.395237 5853 heartbeater.cc:347] Connected to a master server at 127.15.227.254:40137
I20260812 14:44:01.395320 5853 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:01.395480 5853 heartbeater.cc:511] Master 127.15.227.254:40137 requested a full tablet report, sending...
I20260812 14:44:01.395825 5188 ts_manager.cc:194] Registered new tserver with Master: 9662769b99cc4b9fadfa161a9471d0c0 (127.15.227.194:35483)
I20260812 14:44:01.396324 5188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:52481
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:44:01.413558 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:57931/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:40137
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:01.501827 5870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:01.502177 5870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:01.502262 5870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:01.503733 5870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:01.503782 5870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:01.503818 5870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:44:01.505882 5870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.227.212:57931/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:40137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:01.506269 5870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:01.506546 5870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:01.513234 5885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:01.513777 5883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:01.513993 5882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:01.514060 5870 server_base.cc:1034] running on GCE node
I20260812 14:44:01.514380 5870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:01.514608 5870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:01.517858 5870 hybrid_clock.cc:648] HybridClock initialized: now 1786545841517829 us; error 42 us; skew 500 ppm
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:01.520355 5870 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:44:01.520861 5870 webserver.cc:460] Webserver started at http://127.15.227.195:33247/ using document root <none> and password file <none>
I20260812 14:44:01.521095 5870 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:01.521133 5870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:01.521216 5870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:01.535036 5870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "2e132a5c6dff4a4fb874084fe69b8700"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "9dc89a3f83ecfb2e79570ac475af95e6"
server_key_iv: "eed0216ceb2719420938b0bb30bbd79e"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.535547 5870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "2e132a5c6dff4a4fb874084fe69b8700"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "9dc89a3f83ecfb2e79570ac475af95e6"
server_key_iv: "eed0216ceb2719420938b0bb30bbd79e"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.538408 5870 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
I20260812 14:44:01.557436 5896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.557607 5870 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260812 14:44:01.557695 5870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "2e132a5c6dff4a4fb874084fe69b8700"
format_stamp: "Formatted at 2026-08-12 14:44:01 on dist-test-slave-q9h9"
server_key: "9dc89a3f83ecfb2e79570ac475af95e6"
server_key_iv: "eed0216ceb2719420938b0bb30bbd79e"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:01.557780 5870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:01.582497 5870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:01.584811 5870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:01.584997 5870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:01.585263 5870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:01.585695 5870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:01.585800 5870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.585880 5870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:01.585942 5870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:01.592638 5870 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:43047
I20260812 14:44:01.593091 5870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:44:01.595801 6126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:43047 every 8 connection(s)
I20260812 14:44:01.600843 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 5870
I20260812 14:44:01.600945 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:44:01.601135 16271 external_mini_cluster.cc:1428] Setting key b7e2b015a9c6d104537d20ee5f85bfcc
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:01.607070 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.596330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:58221 (local address 127.15.227.254:40137)
0812 14:44:01.596386 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.596388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.596560 (+ 172us) server_negotiation.cc:407] Connection header received
0812 14:44:01.597188 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.597189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.597202 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.597272 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:01.597979 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.598783 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.600257 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.600585 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.600628 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.603836 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:01.603838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:01.603838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:01.603846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:01.605472 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.605956 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.605957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.605958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.606039 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.606338 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.606340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.606340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.606473 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:01.606519 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.606896 (+ 377us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.606978 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:44:01.607821 6130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.596044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40137 (local address 127.15.227.195:58221)
0812 14:44:01.596415 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.596456 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.597062 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.597357 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.597362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.597586 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:01.597864 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.597871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.598900 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.598902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.600149 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.600153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.601387 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.601388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.601528 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.602187 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:01.602197 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:01.603732 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:01.605566 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:01.605572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:01.605575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:01.605864 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:01.606116 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:01.606117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:01.606117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:01.606252 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:01.606567 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:01.606573 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:01.606737 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.607391 (+ 654us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.607587 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":100,"threads_started":1}
I20260812 14:44:01.608407 6127 heartbeater.cc:347] Connected to a master server at 127.15.227.254:40137
I20260812 14:44:01.608505 6127 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:01.608682 6127 heartbeater.cc:511] Master 127.15.227.254:40137 requested a full tablet report, sending...
I20260812 14:44:01.609062 5188 ts_manager.cc:194] Registered new tserver with Master: 2e132a5c6dff4a4fb874084fe69b8700 (127.15.227.195:43047)
I20260812 14:44:01.609560 5188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:58221
I20260812 14:44:01.612833 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:44:01.622068 5607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.614581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40152 (local address 127.15.227.254:40137)
0812 14:44:01.614629 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.614631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.614923 (+ 292us) server_negotiation.cc:407] Connection header received
0812 14:44:01.614970 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.614971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.614981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.615036 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:01.615725 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.616474 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.617513 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.617806 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.617868 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.619283 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:01.619285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:01.619285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:01.619292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:01.620859 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.621239 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.621240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.621240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.621305 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:01.621507 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:01.621507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:01.621508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:01.621622 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:01.621690 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.621888 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.621957 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:44:01.624186 5188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:44:01.628196 5188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:44:01.638562 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.632302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36461 (local address 127.0.0.1:44466)
0812 14:44:01.633850 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.633889 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.633968 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.634312 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.634316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.634319 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:01.634377 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.634378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.635715 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.635716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.637092 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.637095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.638277 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.638278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.638355 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.638369 (+ 14us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.638437 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.638485 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1516,"mutex_wait_us":38,"thread_start_us":75,"threads_started":1}
I20260812 14:44:01.639096 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.632949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.15.227.193:36461)
0812 14:44:01.633747 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.633752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.633972 (+ 220us) server_negotiation.cc:407] Connection header received
0812 14:44:01.634102 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.634105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.634122 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.634206 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:01.634451 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.635562 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.637217 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.638083 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.638170 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.638229 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.638972 (+ 743us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.639015 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":60,"threads_started":1}
I20260812 14:44:01.639551 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.633191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:43047 (local address 127.0.0.1:38744)
0812 14:44:01.634121 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.634129 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.634166 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.634463 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.634464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.634467 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:01.634525 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.634526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.635751 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.635752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.637085 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.637088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.639323 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.639324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.639400 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.639405 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.639436 (+ 31us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.639471 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"mutex_wait_us":37,"thread_start_us":72,"threads_started":1}
I20260812 14:44:01.639868 5481 tablet_service.cc:1467] Processing CreateTablet for tablet 7f338734336140a9b47a9dfce12fc175 (DEFAULT_TABLE table=test-table [id=893ad8f3b5a640aabade71bd078b5364]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:01.640239 5481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f338734336140a9b47a9dfce12fc175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:01.641404 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.632781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38744 (local address 127.15.227.195:43047)
0812 14:44:01.634256 (+ 1475us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.634260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.634268 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:44:01.634300 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.634302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.634316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.634388 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:01.634605 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.635634 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.638071 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.638812 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.641205 (+ 2393us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.641268 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.641293 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.641328 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1442,"thread_start_us":343,"threads_started":1}
I20260812 14:44:01.641988 5968 tablet_service.cc:1467] Processing CreateTablet for tablet 7f338734336140a9b47a9dfce12fc175 (DEFAULT_TABLE table=test-table [id=893ad8f3b5a640aabade71bd078b5364]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:01.642328 5968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f338734336140a9b47a9dfce12fc175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:01.646910 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.633374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:35483 (local address 127.0.0.1:34890)
0812 14:44:01.634637 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.634646 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.634684 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.635007 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.635008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.635011 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:01.635068 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.635070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.639747 (+ 4677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.639748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.640839 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.640842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.646677 (+ 5835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.646678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.646756 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.646760 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.646815 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.646849 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"thread_start_us":66,"threads_started":1}
I20260812 14:44:01.646903 6173 tablet_bootstrap.cc:492] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Bootstrap starting.
I20260812 14:44:01.647330 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.633513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.15.227.194:35483)
0812 14:44:01.634840 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.634844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.634854 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:44:01.634889 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.634891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.634906 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.635148 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:01.635227 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.639215 (+ 3988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.645783 (+ 6568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.646574 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.646685 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.646746 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.647195 (+ 449us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.647236 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1270,"thread_start_us":83,"threads_started":1}
I20260812 14:44:01.647831 6173 tablet_bootstrap.cc:654] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:01.648027 5725 tablet_service.cc:1467] Processing CreateTablet for tablet 7f338734336140a9b47a9dfce12fc175 (DEFAULT_TABLE table=test-table [id=893ad8f3b5a640aabade71bd078b5364]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:01.648150 6173 log.cc:824] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:01.648365 5725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f338734336140a9b47a9dfce12fc175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:01.648489 6169 tablet_bootstrap.cc:492] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71: Bootstrap starting.
I20260812 14:44:01.649401 6169 tablet_bootstrap.cc:654] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:01.650067 6169 log.cc:824] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:01.650853 6173 tablet_bootstrap.cc:492] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: No bootstrap required, opened a new log
I20260812 14:44:01.650919 6173 ts_tablet_manager.cc:1397] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 14:44:01.652797 6173 raft_consensus.cc:348] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.652987 6173 raft_consensus.cc:374] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:01.653021 6173 raft_consensus.cc:729] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e132a5c6dff4a4fb874084fe69b8700, State: Initialized, Role: FOLLOWER
I20260812 14:44:01.653152 6173 consensus_queue.cc:260] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.653466 6173 ts_tablet_manager.cc:1428] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:44:01.653584 6127 heartbeater.cc:503] Master 127.15.227.254:40137 was elected leader, sending a full tablet report...
I20260812 14:44:01.654345 6169 tablet_bootstrap.cc:492] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71: No bootstrap required, opened a new log
I20260812 14:44:01.654410 6169 ts_tablet_manager.cc:1397] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260812 14:44:01.656435 6169 raft_consensus.cc:348] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.656589 6169 raft_consensus.cc:374] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:01.656636 6169 raft_consensus.cc:729] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ceb2ac8c05e437dad8947ba791d6c71, State: Initialized, Role: FOLLOWER
I20260812 14:44:01.656759 6169 consensus_queue.cc:260] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.656967 6169 ts_tablet_manager.cc:1428] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:44:01.657114 5600 heartbeater.cc:503] Master 127.15.227.254:40137 was elected leader, sending a full tablet report...
I20260812 14:44:01.657933 6178 tablet_bootstrap.cc:492] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0: Bootstrap starting.
I20260812 14:44:01.658921 6178 tablet_bootstrap.cc:654] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:01.659227 6178 log.cc:824] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:01.660164 6178 tablet_bootstrap.cc:492] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0: No bootstrap required, opened a new log
I20260812 14:44:01.660238 6178 ts_tablet_manager.cc:1397] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:44:01.662366 6178 raft_consensus.cc:348] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.662523 6178 raft_consensus.cc:374] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:01.662559 6178 raft_consensus.cc:729] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9662769b99cc4b9fadfa161a9471d0c0, State: Initialized, Role: FOLLOWER
I20260812 14:44:01.662674 6178 consensus_queue.cc:260] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.662911 6178 ts_tablet_manager.cc:1428] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:44:01.663045 5853 heartbeater.cc:503] Master 127.15.227.254:40137 was elected leader, sending a full tablet report...
I20260812 14:44:01.699957 6188 raft_consensus.cc:482] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:44:01.700129 6188 raft_consensus.cc:504] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.704327 5855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.700657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36461 (local address 127.15.227.194:60901)
0812 14:44:01.700703 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.700713 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.700745 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.701148 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.701149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.701151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:01.701213 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.701214 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.702147 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.702148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.703311 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.703314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.704091 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.704092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.704165 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.704168 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.704210 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.704257 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 14:44:01.704720 6157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.700720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:60901 (local address 127.15.227.193:36461)
0812 14:44:01.700763 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.700764 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.700793 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:44:01.700817 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.700818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.700827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.700874 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:01.701305 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.702064 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.703432 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.703990 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.704053 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.704061 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.704623 (+ 562us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.704660 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:44:01.704944 5517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f338734336140a9b47a9dfce12fc175" candidate_uuid: "9662769b99cc4b9fadfa161a9471d0c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" is_pre_election: true
I20260812 14:44:01.705060 5517 raft_consensus.cc:2454] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9662769b99cc4b9fadfa161a9471d0c0 in term 0.
I20260812 14:44:01.705181 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.701078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:36845 (local address 127.15.227.195:43047)
0812 14:44:01.701119 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.701120 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.701249 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:44:01.701351 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.701352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.701361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.701387 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:01.701625 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.702340 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.704252 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.704791 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.704838 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.704845 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.705078 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.705120 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:44:01.705408 6190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.700926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:43047 (local address 127.15.227.194:36845)
0812 14:44:01.701176 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.701187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.701278 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.701450 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:01.701451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:01.701452 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:01.701530 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.701532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.703030 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.703031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.704148 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:01.704151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.704922 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.704923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:01.704991 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.704994 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:44:01.705287 (+ 293us) client_negotiation.cc:240] Negotiation successful
0812 14:44:01.705326 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":71,"threads_started":1}
I20260812 14:44:01.705516 5647 leader_election.cc:302] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ceb2ac8c05e437dad8947ba791d6c71, 9662769b99cc4b9fadfa161a9471d0c0; no voters:
I20260812 14:44:01.705713 6018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f338734336140a9b47a9dfce12fc175" candidate_uuid: "9662769b99cc4b9fadfa161a9471d0c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e132a5c6dff4a4fb874084fe69b8700" is_pre_election: true
I20260812 14:44:01.705848 6018 raft_consensus.cc:2454] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9662769b99cc4b9fadfa161a9471d0c0 in term 0.
I20260812 14:44:01.705979 6188 leader_election.cc:288] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ceb2ac8c05e437dad8947ba791d6c71 (127.15.227.193:36461), 2e132a5c6dff4a4fb874084fe69b8700 (127.15.227.195:43047)
I20260812 14:44:01.706094 6188 raft_consensus.cc:2781] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:44:01.706173 6188 raft_consensus.cc:482] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:44:01.706236 6188 raft_consensus.cc:3037] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:01.707286 6188 raft_consensus.cc:504] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.707597 5517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f338734336140a9b47a9dfce12fc175" candidate_uuid: "9662769b99cc4b9fadfa161a9471d0c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ceb2ac8c05e437dad8947ba791d6c71"
I20260812 14:44:01.707595 6018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f338734336140a9b47a9dfce12fc175" candidate_uuid: "9662769b99cc4b9fadfa161a9471d0c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e132a5c6dff4a4fb874084fe69b8700"
I20260812 14:44:01.707669 5517 raft_consensus.cc:3037] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:01.707677 6018 raft_consensus.cc:3037] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:01.708684 6018 raft_consensus.cc:2454] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9662769b99cc4b9fadfa161a9471d0c0 in term 1.
I20260812 14:44:01.709544 5648 leader_election.cc:302] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e132a5c6dff4a4fb874084fe69b8700, 9662769b99cc4b9fadfa161a9471d0c0; no voters:
I20260812 14:44:01.709631 5517 raft_consensus.cc:2454] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9662769b99cc4b9fadfa161a9471d0c0 in term 1.
I20260812 14:44:01.709735 6192 raft_consensus.cc:2781] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:01.709812 6188 leader_election.cc:288] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [CANDIDATE]: Term 1 election: Requested vote from peers 2ceb2ac8c05e437dad8947ba791d6c71 (127.15.227.193:36461), 2e132a5c6dff4a4fb874084fe69b8700 (127.15.227.195:43047)
I20260812 14:44:01.709956 6192 raft_consensus.cc:686] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 1 LEADER]: Becoming Leader. State: Replica: 9662769b99cc4b9fadfa161a9471d0c0, State: Running, Role: LEADER
I20260812 14:44:01.710110 6192 consensus_queue.cc:237] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } }
I20260812 14:44:01.713409 5188 catalog_manager.cc:5495] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9662769b99cc4b9fadfa161a9471d0c0 (127.15.227.194). New cstate: current_term: 1 leader_uuid: "9662769b99cc4b9fadfa161a9471d0c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9662769b99cc4b9fadfa161a9471d0c0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35483 } health_report { overall_health: HEALTHY } } }
I20260812 14:44:01.731094 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.727518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.15.227.194:35483)
0812 14:44:01.727553 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:01.727555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:01.727561 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:44:01.727590 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:01.727591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:01.727608 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:01.727666 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:44:01.727833 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.728542 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.729829 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:01.730162 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:01.730225 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:01.730610 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:44:01.730725 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:44:01.730800 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:01.731009 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 14:44:01.731028 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:44:01.732863 6018 raft_consensus.cc:1261] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 1 FOLLOWER]: Refusing update from remote peer 9662769b99cc4b9fadfa161a9471d0c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:01.733403 6193 consensus_queue.cc:1035] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e132a5c6dff4a4fb874084fe69b8700" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 43047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 12 14:44:01 dist-test-slave-q9h9 krb5kdc[4277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545841, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:01.735361 5517 raft_consensus.cc:1261] T 7f338734336140a9b47a9dfce12fc175 P 2ceb2ac8c05e437dad8947ba791d6c71 [term 1 FOLLOWER]: Refusing update from remote peer 9662769b99cc4b9fadfa161a9471d0c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:01.735630 6193 consensus_queue.cc:1035] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ceb2ac8c05e437dad8947ba791d6c71" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 36461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:01.737375 6197 mvcc.cc:205] Tried to move back new op lower bound from 7317691767735205888 to 7317691767645573120. Current Snapshot: MvccSnapshot[applied={T|T < 7317691767735205888}]
I20260812 14:44:01.737471 6198 mvcc.cc:205] Tried to move back new op lower bound from 7317691767735205888 to 7317691767645573120. Current Snapshot: MvccSnapshot[applied={T|T < 7317691767735205888}]
I20260812 14:44:01.739868 5188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:01.739948 5188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:01.742636 6204 mvcc.cc:205] Tried to move back new op lower bound from 7317691767735205888 to 7317691767645573120. Current Snapshot: MvccSnapshot[applied={T|T < 7317691767735205888}]
I20260812 14:44:01.743069 5188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8daff9ac812a40f983269868f660b936: Sending DeleteTablet for 3 replicas of tablet 7f338734336140a9b47a9dfce12fc175
I20260812 14:44:01.743396 5968 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f338734336140a9b47a9dfce12fc175 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:01 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:38744
I20260812 14:44:01.743906 5481 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f338734336140a9b47a9dfce12fc175 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:01 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:44466
I20260812 14:44:01.744092 5725 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f338734336140a9b47a9dfce12fc175 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:01 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:34890
I20260812 14:44:01.744678 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 5354
I20260812 14:44:01.795949 6213 tablet_replica.cc:330] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0: stopping tablet replica
I20260812 14:44:01.797261 6213 raft_consensus.cc:2229] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:44:01.797506 6213 raft_consensus.cc:2258] T 7f338734336140a9b47a9dfce12fc175 P 9662769b99cc4b9fadfa161a9471d0c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:01.797672 6209 tablet_replica.cc:330] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: stopping tablet replica
I20260812 14:44:01.797897 6209 raft_consensus.cc:2229] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:44:01.798085 6209 raft_consensus.cc:2258] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:01.798631 6209 ts_tablet_manager.cc:1905] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:01.800290 6209 ts_tablet_manager.cc:1918] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:01.800412 6209 log.cc:1196] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/7f338734336140a9b47a9dfce12fc175
I20260812 14:44:01.801044 6209 ts_tablet_manager.cc:1939] T 7f338734336140a9b47a9dfce12fc175 P 2e132a5c6dff4a4fb874084fe69b8700: Deleting consensus metadata
I20260812 14:44:01.801535 5154 catalog_manager.cc:4841] TS 2e132a5c6dff4a4fb874084fe69b8700 (127.15.227.195:43047): tablet 7f338734336140a9b47a9dfce12fc175 (table test-table [id=893ad8f3b5a640aabade71bd078b5364]) successfully deleted
W20260812 14:44:01.803079 5153 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:36461 (error 108)
W20260812 14:44:01.803746 6155 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:44:01.803467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:36461 (local address 127.0.0.1:44474)
0812 14:44:01.803552 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.803563 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:01.803598 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:01.803698 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:36461: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":65}
W20260812 14:44:01.803864 5153 catalog_manager.cc:4584] TS 2ceb2ac8c05e437dad8947ba791d6c71 (127.15.227.193:36461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f338734336140a9b47a9dfce12fc175: Network error: Client connection negotiation failed: client connection to 127.15.227.193:36461: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:44:01.808051 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 5611
I20260812 14:44:01.824425 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:01.824273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:35483 (local address 127.0.0.1:34906)
0812 14:44:01.824328 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:01.824383 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.194:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:44:01.824697 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 5870
I20260812 14:44:01.852944 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 5125
I20260812 14:44:01.865517 16271 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 14:44:02.212961 16271 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 14:44:02.213069 16271 mini_ranger.cc:66] Stopping Ranger...
I20260812 14:44:02.556602 16271 mini_ranger.cc:68] Stopped Ranger
2026-08-12 14:44:02.556 UTC [4338] LOG: received smart shutdown request
2026-08-12 14:44:02.560 UTC [4338] LOG: background worker "logical replication launcher" (PID 4354) exited with exit code 1
2026-08-12 14:44:02.560 UTC [4345] LOG: shutting down
2026-08-12 14:44:02.672 UTC [4338] LOG: database system is shut down
2026-08-12T14:44:02Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44643 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:44:02 dist-test-slave-q9h9 krb5kdc[6231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:44:02 dist-test-slave-q9h9 krb5kdc[6231](info): set up 2 sockets
Aug 12 14:44:02 dist-test-slave-q9h9 krb5kdc[6231](info): commencing operation
krb5kdc: starting...
W20260812 14:44:03.943943 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:44:04 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545844, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:44:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/rangeradmin_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/rangeradmin_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/rangerlookup_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/rangerlookup_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
I20260812 14:44:04.084048 16271 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskN3u55Q/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/postgres -l logfile start
2026-08-12 14:44:06.110 UTC [6294] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 14:44:06.110 UTC [6294] LOG: listening on IPv4 address "127.15.227.212", port 48741
2026-08-12 14:44:06.112 UTC [6294] LOG: listening on Unix socket "/tmp/.s.PGSQL.48741"
2026-08-12 14:44:06.130 UTC [6297] LOG: database system was shut down at 2026-08-12 14:44:05 UTC
2026-08-12 14:44:06.133 UTC [6294] LOG: database system is ready to accept connections
I20260812 14:44:07.239382 16271 mini_postgres.cc:96] Postgres bound to 48741
2026-08-12 14:44:07.246 UTC [6308] FATAL: database "slave" does not exist
127.15.227.212:48741 - accepting connections
I20260812 14:44:07.246477 16271 mini_ranger.cc:161] Starting Ranger...
I20260812 14:44:07.260792 16271 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 14:44:07.586794 16271 mini_ranger.cc:87] Created ranger Postgres database
I20260812 14:44:07.586865 16271 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-admin
2026-08-12 14:44:07,850 [I] DB FLAVOR :POSTGRES
2026-08-12 14:44:07,850 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:44:07,850 [I] Checking connection..
2026-08-12 14:44:07,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 14:44:08,100 [I] Checking connection passed.
2026-08-12 14:44:08,100 [I] --------- Verifying version history table ---------
2026-08-12 14:44:08,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:44:08,339 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:44:08,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:44:08,583 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 14:44:08,583 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 14:44:08,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 14:44:08.825 UTC [6418] WARNING: there is no transaction in progress
2026-08-12 14:44:08,833 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 14:44:08,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 14:44:09,085 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 14:44:09,085 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:44:09,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 14:44:09,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 14:44:09,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-q9h9.c.gcp-upstream.internal','N') ;"
2026-08-12 14:44:09,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:44:10,056 [I] Table x_portal_user does not exist in database ranger
2026-08-12 14:44:10,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:44:10,297 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 14:44:10,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:44:10,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:44:10,798 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 14:44:10,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 14:44:11.616 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.631 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.645 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.657 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.671 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.688 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.745 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.750 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.762 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.771 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.779 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.791 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.799 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.804 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.809 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:11.815 UTC [6602] WARNING: there is no transaction in progress
2026-08-12 14:44:12,320 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 14:44:12,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 14:44:12,548 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 14:44:12,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 14:44:12,778 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 14:44:12,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 14:44:13,024 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 14:44:13,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 14:44:13,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 14:44:13,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-q9h9.c.gcp-upstream.internal';"
2026-08-12 14:44:13,748 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 14:44:13,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 14:44:13,997 [I] DB_PATCHES have already been applied
I20260812 14:44:14.002650 16271 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-admin:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskN3u55Q/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 14:44:14.002713 16271 mini_ranger.cc:193] Using host: 127.15.227.212
I20260812 14:44:14.006275 16271 mini_ranger.cc:227] Ranger admin URL: http://127.15.227.212:49369
Aug 12, 2026 2:44:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:44:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:44:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 2:44:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 2:44:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 2:44:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 2:44:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 14:44:15.305451 16271 mini_ranger.cc:160] Time spent starting Ranger: real 8.059s user 0.000s sys 0.009s
Aug 12, 2026 2:44:34 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 2:44:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 2:44:34 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 2:44:34 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 2:44:35 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 14:44:36.028110 16271 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.227.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.227.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/rangerkms_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.227.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/rangerkms_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.227.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.227.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.227.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/HTTP_127.15.227.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 14:44:36 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545876, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 14:44:36.095844 16271 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 14:44:36.104175 16271 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 14:44:36.717337 16271 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 14:44:36.717411 16271 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms
I20260812 14:44:36.717427 16271 mini_ranger_kms.cc:227] Using postgres at 127.15.227.212:48741
2026-08-12 14:44:36,862 [I] DB FLAVOR :POSTGRES
2026-08-12 14:44:36,862 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 14:44:36,862 [I] Checking connection
2026-08-12 14:44:36,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 14:44:37,096 [I] connection success
2026-08-12 14:44:37,096 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 14:44:37,096 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 14:44:37,096 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 14:44:37,316 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 14:44:37,317 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 14:44:37,317 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 14:44:37,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.227.212:48741/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 14:44:37,550 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 14:44:37.982455 16271 mini_ranger_kms.cc:310] Created kms service
I20260812 14:44:38.185256 16271 mini_ranger_kms.cc:326] Created kudu user
I20260812 14:44:38.227111 16271 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 14:44:38 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545876, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:38.608182 16271 mini_ranger_kms.cc:384] Added ranger policy
I20260812 14:44:38.608285 16271 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms:/tmp/dist-test-taskN3u55Q/build/release/bin/postgresql.jar:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskN3u55Q/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 14:44:38.608304 16271 mini_ranger_kms.cc:240] Using host: 127.15.227.212
I20260812 14:44:38.612246 16271 mini_ranger_kms.cc:276] Ranger KMS PID: 6951
I20260812 14:44:38.612320 16271 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.227.212:38803
Aug 12, 2026 2:44:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 2:44:39 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 2:44:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 2:44:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 2:44:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 2:44:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 14:44:40 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786545880, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.227.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 14:44:41 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545880, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.227.212@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:41.256626 16271 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.161s user 0.004s sys 0.125s
I20260812 14:44:41.256719 16271 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 14:44:41.256762 16271 mini_ranger_kms.cc:401] 127.15.227.212:38803/kms/v1/keys
Aug 12 14:44:41 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545880, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.227.212@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
Aug 12 14:44:43 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:44:43.848665 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43061
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:43061
--ranger_config_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:43.927685 7062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:43.927879 7062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:43.927899 7062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:43.929008 7062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:43.929045 7062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:43.929059 7062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:43.929069 7062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:43.929080 7062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:43.930599 7062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.227.212:38803/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:43061
--ranger_config_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:43.930820 7062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:43.931016 7062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:43.936641 7069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:43.936640 7067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:43.936754 7073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:43.937002 7062 server_base.cc:1034] running on GCE node
I20260812 14:44:43.937206 7062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:43.937471 7062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:43.938611 7062 hybrid_clock.cc:648] HybridClock initialized: now 1786545883938593 us; error 36 us; skew 500 ppm
Aug 12 14:44:43 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545883, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:43.940876 7062 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:43.941285 7062 webserver.cc:460] Webserver started at http://127.15.227.254:42395/ using document root <none> and password file <none>
I20260812 14:44:43.941483 7062 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:43.941519 7062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:43.941636 7062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:43 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545883, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:44.000551 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "7ccce921fcb344959c31eb072b04c504"
format_stamp: "Formatted at 2026-08-12 14:44:43 on dist-test-slave-q9h9"
server_key: "ff9b571bbe597c4668d1aea5b0723955"
server_key_iv: "d90699083bf994c0708592298e300898"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:44.000991 7062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "7ccce921fcb344959c31eb072b04c504"
format_stamp: "Formatted at 2026-08-12 14:44:43 on dist-test-slave-q9h9"
server_key: "ff9b571bbe597c4668d1aea5b0723955"
server_key_iv: "d90699083bf994c0708592298e300898"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:44.002828 7062 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260812 14:44:44.017203 7092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:44.017575 7062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:44.017855 7062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "7ccce921fcb344959c31eb072b04c504"
format_stamp: "Formatted at 2026-08-12 14:44:43 on dist-test-slave-q9h9"
server_key: "ff9b571bbe597c4668d1aea5b0723955"
server_key_iv: "d90699083bf994c0708592298e300898"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:44.017938 7062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:44.035718 7062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:44.037518 7062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:44.037643 7062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:44.042121 7062 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:43061
I20260812 14:44:44.042196 7179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:43061 every 8 connection(s)
I20260812 14:44:44.042523 7062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:44.043160 7180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:44.043591 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 7062
I20260812 14:44:44.043695 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:44:44.043869 16271 external_mini_cluster.cc:1428] Setting key d5b17d319473566c42fb848f9a58137f
I20260812 14:44:44.045979 7180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Bootstrap starting.
I20260812 14:44:44.046700 7180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:44.046983 7180 log.cc:824] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:44.047698 7180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: No bootstrap required, opened a new log
I20260812 14:44:44.049759 7180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:44.049887 7180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:44.049921 7180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ccce921fcb344959c31eb072b04c504, State: Initialized, Role: FOLLOWER
I20260812 14:44:44.050094 7180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:44.050163 7180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:44:44.050194 7180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:44:44.050252 7180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:44.050746 7180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:44.050853 7180 leader_election.cc:302] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ccce921fcb344959c31eb072b04c504; no voters:
I20260812 14:44:44.051003 7180 leader_election.cc:288] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:44:44.051044 7189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:44.051268 7189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 LEADER]: Becoming Leader. State: Replica: 7ccce921fcb344959c31eb072b04c504, State: Running, Role: LEADER
Aug 12 14:44:44 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:44.051424 7189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:44.051712 7189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ccce921fcb344959c31eb072b04c504. Latest consensus state: current_term: 1 leader_uuid: "7ccce921fcb344959c31eb072b04c504" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } } }
I20260812 14:44:44.051787 7189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:44.051304 7180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:44.051723 7192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ccce921fcb344959c31eb072b04c504" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } } }
I20260812 14:44:44.052006 7192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:44.053251 7200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:44.053359 7180 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:44:44.053800 7200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:44:44.055233 7200 catalog_manager.cc:1347] Generated new cluster ID: 786b6cefd86149a2b5cb84a65c03a92f
I20260812 14:44:44.055285 7200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:44:44.055500 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:44.044381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.15.227.254:43061)
0812 14:44:44.044536 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:44.044545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:44.044741 (+ 196us) server_negotiation.cc:407] Connection header received
0812 14:44:44.045257 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:44.045266 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:44.045340 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:44.045486 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:44.046207 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:44.047413 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:44.048496 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:44.049097 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:44.049204 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:44.051887 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:44.051891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:44.051892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:44.051903 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:44.053627 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:44.054037 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:44.054041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:44.054041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:44.054139 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:44.054350 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:44.054350 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:44.054351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:44.054628 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:44.054777 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:44.055191 (+ 414us) server_negotiation.cc:299] Negotiation successful
0812 14:44:44.055312 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260812 14:44:44.081226 7200 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:44:44.082003 7200 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:44:44.085851 7200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Generated new TSK 0
I20260812 14:44:44.086112 7200 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:44:44.778772 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43061
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:44.884212 7270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:44.884393 7270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:44.884423 7270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:44.885488 7270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:44.885525 7270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:44.885557 7270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:44:44.887329 7270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.227.212:38803/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:44.887543 7270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:44.887734 7270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:44.892613 7286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:44.892622 7283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:44.892741 7270 server_base.cc:1034] running on GCE node
W20260812 14:44:44.892803 7281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:44.893047 7270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:44.893299 7270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:44.894430 7270 hybrid_clock.cc:648] HybridClock initialized: now 1786545884894418 us; error 30 us; skew 500 ppm
Aug 12 14:44:44 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545884, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:44.896860 7270 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:44:44.897279 7270 webserver.cc:460] Webserver started at http://127.15.227.193:39115/ using document root <none> and password file <none>
I20260812 14:44:44.897476 7270 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:44.897523 7270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:44.897643 7270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:44 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545884, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:44.914705 7270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "74cfa025dce948a88d90bb5ed1503810"
format_stamp: "Formatted at 2026-08-12 14:44:44 on dist-test-slave-q9h9"
server_key: "0ed51a32fc797f2b48f420bc96a1fafa"
server_key_iv: "7bc1f1149693bf5f7b58872c50550ba9"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:44.915153 7270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "74cfa025dce948a88d90bb5ed1503810"
format_stamp: "Formatted at 2026-08-12 14:44:44 on dist-test-slave-q9h9"
server_key: "0ed51a32fc797f2b48f420bc96a1fafa"
server_key_iv: "7bc1f1149693bf5f7b58872c50550ba9"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:44.916911 7270 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260812 14:44:44.938159 7299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:44.939460 7270 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260812 14:44:44.939571 7270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "74cfa025dce948a88d90bb5ed1503810"
format_stamp: "Formatted at 2026-08-12 14:44:44 on dist-test-slave-q9h9"
server_key: "0ed51a32fc797f2b48f420bc96a1fafa"
server_key_iv: "7bc1f1149693bf5f7b58872c50550ba9"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:44.939656 7270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:44.965253 7270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:44.967546 7270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:44.967751 7270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:44.968004 7270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:44.968401 7270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:44.968492 7270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:44.968596 7270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:44.968672 7270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:44.974864 7270 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:42439
I20260812 14:44:44.975124 7528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:42439 every 8 connection(s)
I20260812 14:44:44.975317 7270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
Aug 12 14:44:44 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545884, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:44.984414 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 7270
I20260812 14:44:44.984520 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:44:44.984711 16271 external_mini_cluster.cc:1428] Setting key 24ff3018d653550162de0a96bc8bd0d0
I20260812 14:44:44.990497 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:44.976005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43061 (local address 127.15.227.193:46653)
0812 14:44:44.977521 (+ 1516us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:44.977576 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:44.978210 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:44.978634 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:44.978639 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:44.978911 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:44.979171 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:44.979178 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:44.980172 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:44.980173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:44.981477 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:44.981480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:44.982075 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:44.982077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:44.982417 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:44.982904 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:44.982913 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:44.984517 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:44.988958 (+ 4441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:44.988964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:44.988966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:44.989266 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:44.989474 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:44.989475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:44.989475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:44.989604 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:44.989872 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:44.989878 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:44.990044 (+ 166us) client_negotiation.cc:769] Sending connection context
0812 14:44:44.990144 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 14:44:44.990282 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1479,"thread_start_us":90,"threads_started":1}
I20260812 14:44:44.991024 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:44.976310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46653 (local address 127.15.227.254:43061)
0812 14:44:44.978393 (+ 2083us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:44.978395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:44.978402 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:44:44.978427 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:44.978428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:44.978440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:44.978499 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:44.979294 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:44.980070 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:44.981585 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:44.981898 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:44.981960 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:44.984683 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:44.984685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:44.984685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:44.984692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:44.988870 (+ 4178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:44.989348 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:44.989350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:44.989350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:44.989414 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:44.989668 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:44.989669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:44.989669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:44.989796 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:44.989849 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:44.990835 (+ 986us) server_negotiation.cc:299] Negotiation successful
0812 14:44:44.990921 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2048,"thread_start_us":76,"threads_started":1}
I20260812 14:44:44.992013 7530 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43061
I20260812 14:44:44.992214 7530 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:44.992492 7530 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:44.993043 7120 ts_manager.cc:194] Registered new tserver with Master: 74cfa025dce948a88d90bb5ed1503810 (127.15.227.193:42439)
I20260812 14:44:44.993934 7120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:46653
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:44:45.022995 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43061
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 14:44:45.143517 7541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:45.143766 7541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:45.143800 7541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:45.145418 7541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:45.145509 7541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:45.145562 7541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:44:45.147652 7541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.227.212:38803/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:45.148072 7541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:45.148373 7541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:45.154721 7555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:45.154735 7552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:45.155040 7551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:45.155488 7541 server_base.cc:1034] running on GCE node
I20260812 14:44:45.155711 7541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:45.155907 7541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:45.157050 7541 hybrid_clock.cc:648] HybridClock initialized: now 1786545885157032 us; error 37 us; skew 500 ppm
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:45.159628 7541 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:44:45.160033 7541 webserver.cc:460] Webserver started at http://127.15.227.194:40005/ using document root <none> and password file <none>
I20260812 14:44:45.160248 7541 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:45.160300 7541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:45.160437 7541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:45.174695 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "1e47cc55f3dc4cdd856b0d4008301844"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "2c5853f62d84ea1cdba3ef9bd895151a"
server_key_iv: "41dbbd33220d9b4676374bc6ffcabf46"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:45.175166 7541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "1e47cc55f3dc4cdd856b0d4008301844"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "2c5853f62d84ea1cdba3ef9bd895151a"
server_key_iv: "41dbbd33220d9b4676374bc6ffcabf46"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:45.177158 7541 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:45.194588 7567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:45.194814 7541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:45.194908 7541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "1e47cc55f3dc4cdd856b0d4008301844"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "2c5853f62d84ea1cdba3ef9bd895151a"
server_key_iv: "41dbbd33220d9b4676374bc6ffcabf46"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:45.194983 7541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:45.216758 7541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:45.218945 7541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:45.219086 7541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:45.219296 7541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:45.219740 7541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:45.219841 7541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:45.219913 7541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:45.219975 7541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:45.226941 7541 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:45631
I20260812 14:44:45.227430 7541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:44:45.229417 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 7541
I20260812 14:44:45.229517 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:44:45.229707 16271 external_mini_cluster.cc:1428] Setting key 067279dc07aec036f189c5b1f2bf3f30
I20260812 14:44:45.234404 7787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:45631 every 8 connection(s)
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:45.254941 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:45.234527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:45561 (local address 127.15.227.254:43061)
0812 14:44:45.234572 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:45.234574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:45.242262 (+ 7688us) server_negotiation.cc:407] Connection header received
0812 14:44:45.243028 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:45.243030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:45.243046 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:45.243076 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:45.243968 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.244847 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.248597 (+ 3750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.248930 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.248979 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:45.251628 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:45.251630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:45.251630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:45.251638 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:45.253276 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:45.253769 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:45.253770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:45.253770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:45.253832 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:45.254136 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:45.254137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:45.254137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:45.254274 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:45.254327 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:45.254729 (+ 402us) server_negotiation.cc:299] Negotiation successful
0812 14:44:45.254846 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:44:45.255676 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:45.234246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43061 (local address 127.15.227.194:45561)
0812 14:44:45.242113 (+ 7867us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:45.242166 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:45.242854 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:45.243312 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:45.243317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:45.243546 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:45.243845 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:45.243853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.245131 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.245132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:45.248497 (+ 3365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:45.248500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.249084 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.249085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:45.249224 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:45.249861 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:45.249871 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:45.251511 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:45.253370 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:45.253377 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:45.253379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:45.253675 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:45.253901 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:45.253902 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:45.253902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:45.254054 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:45.254362 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:45.254370 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:45.254526 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 14:44:45.255275 (+ 749us) client_negotiation.cc:240] Negotiation successful
0812 14:44:45.255437 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7814,"thread_start_us":94,"threads_started":1}
I20260812 14:44:45.256323 7789 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43061
I20260812 14:44:45.256417 7789 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:45.256596 7789 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:45.257014 7120 ts_manager.cc:194] Registered new tserver with Master: 1e47cc55f3dc4cdd856b0d4008301844 (127.15.227.194:45631)
I20260812 14:44:45.257505 7120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:45561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:44:45.278645 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43061
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:45.397789 7803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:45.398120 7803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:45.398205 7803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:45.399870 7803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:45.399974 7803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:45.400053 7803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:44:45.402045 7803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.227.212:38803/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:45.402470 7803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:45.402776 7803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:45.409144 7823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:45.409325 7818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:45.409519 7829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:45.414196 7803 server_base.cc:1034] running on GCE node
I20260812 14:44:45.414413 7803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:45.414690 7803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:45.415861 7803 hybrid_clock.cc:648] HybridClock initialized: now 1786545885415835 us; error 41 us; skew 500 ppm
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:45.419111 7803 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:44:45.419597 7803 webserver.cc:460] Webserver started at http://127.15.227.195:41633/ using document root <none> and password file <none>
I20260812 14:44:45.419869 7803 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:45.419926 7803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:45.420053 7803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:45.453087 7803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "986b70deba3c4522a2c09b35addbaa93"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "ce0e861316f396f24a2033569a5f6a3e"
server_key_iv: "238f6c4c00ca6e9709b7e795159f9d4c"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:45.453526 7803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "986b70deba3c4522a2c09b35addbaa93"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "ce0e861316f396f24a2033569a5f6a3e"
server_key_iv: "238f6c4c00ca6e9709b7e795159f9d4c"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:45.456754 7803 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260812 14:44:45.472110 7844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:45.472364 7803 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:45.472460 7803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "986b70deba3c4522a2c09b35addbaa93"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "ce0e861316f396f24a2033569a5f6a3e"
server_key_iv: "238f6c4c00ca6e9709b7e795159f9d4c"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:45.472543 7803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:45.490553 7803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:45.492666 7803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:45.492795 7803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:45.493022 7803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:45.493348 7803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:45.493393 7803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:45.493425 7803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:45.493446 7803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:45.499775 7803 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:37141
I20260812 14:44:45.500162 7803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:44:45.503451 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 7803
I20260812 14:44:45.503549 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:44:45.503737 16271 external_mini_cluster.cc:1428] Setting key e424ac393cd9bcd8600a197cb0754014
I20260812 14:44:45.511324 8060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:37141 every 8 connection(s)
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:45.524473 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:45.511954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:40649 (local address 127.15.227.254:43061)
0812 14:44:45.512016 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:45.512018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:45.512226 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:44:45.515505 (+ 3279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:45.515506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:45.515518 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:45.515546 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:45.516332 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.517082 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.518544 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.518839 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.518878 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:45.521455 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:45.521456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:45.521457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:45.521463 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:45.523056 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:45.523494 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:45.523495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:45.523496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:45.523553 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:45.523812 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:45.523813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:45.523813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:45.523932 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:45.523970 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:45.524300 (+ 330us) server_negotiation.cc:299] Negotiation successful
0812 14:44:45.524383 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:44:45.525218 8067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:45.511939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43061 (local address 127.15.227.195:40649)
0812 14:44:45.512108 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:45.512149 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:45.515394 (+ 3245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:45.515736 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:45.515740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:45.515969 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:45.516228 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:45.516234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.517195 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.517196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:45.518449 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:45.518452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:45.518975 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:45.518976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:45.519115 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:45.519647 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:45.519669 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:45.521359 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:45.523148 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:45.523155 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:45.523157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:45.523413 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:45.523615 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:45.523616 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:45.523616 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:45.523745 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:45.524011 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:45.524016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:45.524161 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 14:44:45.524798 (+ 637us) client_negotiation.cc:240] Negotiation successful
0812 14:44:45.524946 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260812 14:44:45.526298 8061 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43061
I20260812 14:44:45.526381 8061 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:45.526553 8061 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:45.526937 7120 ts_manager.cc:194] Registered new tserver with Master: 986b70deba3c4522a2c09b35addbaa93 (127.15.227.195:37141)
I20260812 14:44:45.527410 7120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:40649
I20260812 14:44:45.536633 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:44:45.537338 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 7270
I20260812 14:44:45.607952 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 7541
I20260812 14:44:45.628444 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 7803
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:45.646476 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 7062
I20260812 14:44:45.697281 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43061
--webserver_interface=127.15.227.254
--webserver_port=42395
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:43061
--ranger_config_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:45.777441 8069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:45.777643 8069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:45.777664 8069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:45.778815 8069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:45.778855 8069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:45.778868 8069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:45.778880 8069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:45.778904 8069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:45.780436 8069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:43061
--ranger_config_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:45.780654 8069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:45.780856 8069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:45.785490 8087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:45.785498 8078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:45.785532 8069 server_base.cc:1034] running on GCE node
W20260812 14:44:45.785657 8084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:45.785879 8069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:45.786185 8069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:45.787325 8069 hybrid_clock.cc:648] HybridClock initialized: now 1786545885787307 us; error 36 us; skew 500 ppm
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:45.789587 8069 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:45.789970 8069 webserver.cc:460] Webserver started at http://127.15.227.254:42395/ using document root <none> and password file <none>
I20260812 14:44:45.790194 8069 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:45.790254 8069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:45.864529 8069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:44:45.865201 8105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:45.865346 8069 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:45.865434 8069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "7ccce921fcb344959c31eb072b04c504"
format_stamp: "Formatted at 2026-08-12 14:44:43 on dist-test-slave-q9h9"
server_key: "ff9b571bbe597c4668d1aea5b0723955"
server_key_iv: "d90699083bf994c0708592298e300898"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:45.865787 8069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:45.888530 8069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:45.890367 8069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:45.890489 8069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:45.894640 8069 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:43061
I20260812 14:44:45.894683 8220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:43061 every 8 connection(s)
I20260812 14:44:45.895032 8069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:45.896195 8222 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:44:45.896819 8222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Bootstrap starting.
I20260812 14:44:45.903107 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 8069
I20260812 14:44:45.903450 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:42439
--local_ip_for_outbound_sockets=127.15.227.193
--tserver_master_addrs=127.15.227.254:43061
--webserver_port=39115
--webserver_interface=127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:45.904757 8222 log.cc:824] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:45.906436 8222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:44:45.906682 8222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Bootstrap complete.
I20260812 14:44:45.908841 8222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:45.909060 8222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ccce921fcb344959c31eb072b04c504, State: Initialized, Role: FOLLOWER
I20260812 14:44:45.909202 8222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:45.909267 8222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:44:45.909308 8222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:44:45.909351 8222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:44:45.910334 8222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:45.910420 8222 leader_election.cc:302] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ccce921fcb344959c31eb072b04c504; no voters:
I20260812 14:44:45.910563 8222 leader_election.cc:288] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:44:45.910609 8231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:44:45.910781 8231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [term 2 LEADER]: Becoming Leader. State: Replica: 7ccce921fcb344959c31eb072b04c504, State: Running, Role: LEADER
I20260812 14:44:45.910820 8222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:45.910935 8231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } }
I20260812 14:44:45.911592 8234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7ccce921fcb344959c31eb072b04c504" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } } }
I20260812 14:44:45.911629 8233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ccce921fcb344959c31eb072b04c504. Latest consensus state: current_term: 2 leader_uuid: "7ccce921fcb344959c31eb072b04c504" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ccce921fcb344959c31eb072b04c504" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43061 } } }
I20260812 14:44:45.911718 8233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:45.911676 8234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:45.912055 8253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:45.912313 8222 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 14:44:45.912611 8253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:44:45.914644 8253 catalog_manager.cc:1259] Loaded cluster ID: 786b6cefd86149a2b5cb84a65c03a92f
I20260812 14:44:45.914824 8253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:44:45.916891 8253 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:44:45.917142 8253 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Loaded TSK: 0
I20260812 14:44:45.917444 8253 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:44:45.990892 8230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:45.991212 8230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:45.991313 8230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:45.992486 8230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:45.992534 8230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:45.992570 8230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:44:45.994077 8230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:42439
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=39115
--tserver_master_addrs=127.15.227.254:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:45.994292 8230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:45.994525 8230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:45.999495 8302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:45.999800 8305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:46.000033 8300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:46.000591 8230 server_base.cc:1034] running on GCE node
I20260812 14:44:46.000783 8230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:46.001006 8230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:46.002157 8230 hybrid_clock.cc:648] HybridClock initialized: now 1786545886002141 us; error 25 us; skew 500 ppm
Aug 12 14:44:45 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:46.004638 8230 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:44:46.005015 8230 webserver.cc:460] Webserver started at http://127.15.227.193:39115/ using document root <none> and password file <none>
I20260812 14:44:46.005232 8230 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:46.005306 8230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:46.024894 8230 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260812 14:44:46.025763 8323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.025956 8230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:44:46.026079 8230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "74cfa025dce948a88d90bb5ed1503810"
format_stamp: "Formatted at 2026-08-12 14:44:44 on dist-test-slave-q9h9"
server_key: "0ed51a32fc797f2b48f420bc96a1fafa"
server_key_iv: "7bc1f1149693bf5f7b58872c50550ba9"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:46.026535 8230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:46.048636 8230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:46.050827 8230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:46.051023 8230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:46.051297 8230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:46.051668 8230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:46.051774 8230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.051873 8230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:46.051945 8230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.058666 8230 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:42439
I20260812 14:44:46.059033 8230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:44:46.059412 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 8230
I20260812 14:44:46.059666 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:45631
--local_ip_for_outbound_sockets=127.15.227.194
--tserver_master_addrs=127.15.227.254:43061
--webserver_port=40005
--webserver_interface=127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:46.073621 8566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:42439 every 8 connection(s)
I20260812 14:44:46.073956 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:46.061730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43061 (local address 127.15.227.193:59257)
0812 14:44:46.062207 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:46.062259 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:46.062828 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:46.063469 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:46.063473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:46.063678 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:46.063917 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:46.063924 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.065611 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.065613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:46.066872 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:46.066875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.067785 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.067786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:46.067914 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:46.068495 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:46.068505 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:46.070103 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:46.072281 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:46.072287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:46.072289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:46.072566 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:46.072838 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:46.072839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:46.072839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:46.072955 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:46.073383 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:46.073389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:46.073523 (+ 134us) client_negotiation.cc:769] Sending connection context
0812 14:44:46.073608 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:44:46.073741 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":77,"threads_started":1}
I20260812 14:44:46.074564 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:46.061828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59257 (local address 127.15.227.254:43061)
0812 14:44:46.062125 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:46.062134 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:46.062979 (+ 845us) server_negotiation.cc:407] Connection header received
0812 14:44:46.063125 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:46.063131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:46.063184 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:46.063310 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:46.064038 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.065493 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.066970 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.067579 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.067680 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:46.070227 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:46.070231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:46.070231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:46.070256 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:46.072172 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.072659 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.072662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.072662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.072770 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.073016 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.073017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.073018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.073288 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:46.073357 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:46.074234 (+ 877us) server_negotiation.cc:299] Negotiation successful
0812 14:44:46.074353 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":102,"threads_started":1}
I20260812 14:44:46.075270 8568 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43061
I20260812 14:44:46.075359 8568 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.075533 8568 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
W20260812 14:44:46.076373 8568 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:43061 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:44:46.076443 8568 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.076506 8568 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:46.076836 8568 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.076910 8568 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
W20260812 14:44:46.077193 8568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:44:46.175386 8570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:46.175640 8570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:46.175678 8570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:46.177276 8570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:46.177335 8570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:46.177372 8570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:44:46.179455 8570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:45631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=40005
--tserver_master_addrs=127.15.227.254:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:46.179674 8570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:46.179921 8570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:46.185194 8582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:46.185191 8584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:46.185412 8570 server_base.cc:1034] running on GCE node
W20260812 14:44:46.185504 8590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:46.185678 8570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:46.185909 8570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:46.187040 8570 hybrid_clock.cc:648] HybridClock initialized: now 1786545886187019 us; error 40 us; skew 500 ppm
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:46.189510 8570 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:44:46.190007 8570 webserver.cc:460] Webserver started at http://127.15.227.194:40005/ using document root <none> and password file <none>
I20260812 14:44:46.190274 8570 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:46.190343 8570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:46.211849 8570 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260812 14:44:46.214594 8602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.214761 8570 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260812 14:44:46.214870 8570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "1e47cc55f3dc4cdd856b0d4008301844"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "2c5853f62d84ea1cdba3ef9bd895151a"
server_key_iv: "41dbbd33220d9b4676374bc6ffcabf46"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:46.215308 8570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:46.248679 8570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:46.250871 8570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:46.251034 8570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:46.251288 8570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:46.251597 8570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:46.251634 8570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.251663 8570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:46.251679 8570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.258656 8570 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:45631
I20260812 14:44:46.259069 8570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:44:46.259543 8862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:45631 every 8 connection(s)
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:46.267519 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 8570
I20260812 14:44:46.267798 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.227.212:38803/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:37141
--local_ip_for_outbound_sockets=127.15.227.195
--tserver_master_addrs=127.15.227.254:43061
--webserver_port=41633
--webserver_interface=127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:33225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:46.269724 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:46.259811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:56083 (local address 127.15.227.254:43061)
0812 14:44:46.259866 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:46.259869 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:46.260077 (+ 208us) server_negotiation.cc:407] Connection header received
0812 14:44:46.260649 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:46.260651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:46.260665 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:46.260736 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:46.261398 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.262165 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.263559 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.263862 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.263933 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:46.266324 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:46.266326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:46.266327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:46.266335 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:46.267978 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.268553 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.268555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.268555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.268619 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.268920 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.268922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.268922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.269058 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:46.269105 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:46.269547 (+ 442us) server_negotiation.cc:299] Negotiation successful
0812 14:44:46.269635 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:44:46.270279 8874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:46.259795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43061 (local address 127.15.227.194:56083)
0812 14:44:46.259945 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:46.259984 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:46.260539 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:46.260807 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:46.260811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:46.261020 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:46.261286 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:46.261293 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.262256 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.262258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:46.263468 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:46.263471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.263969 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.263970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:46.264106 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:46.264585 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:46.264594 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:46.266222 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:46.268073 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:46.268079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:46.268082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:46.268429 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:46.268694 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:46.268695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:46.268695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:46.268833 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:46.269170 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:46.269176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:46.269376 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 14:44:46.269862 (+ 486us) client_negotiation.cc:240] Negotiation successful
0812 14:44:46.270048 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260812 14:44:46.270774 8866 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43061
I20260812 14:44:46.270874 8866 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.271065 8866 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
W20260812 14:44:46.271798 8866 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:43061 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:44:46.271865 8866 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.271932 8866 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:46.272245 8866 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.272317 8866 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
W20260812 14:44:46.272596 8866 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:44:46.388439 8877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:46.388691 8877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:46.388729 8877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:46.390363 8877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:46.390422 8877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:46.390458 8877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:44:46.392531 8877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:33225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.227.212:38803/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:37141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=41633
--tserver_master_addrs=127.15.227.254:43061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:46.392751 8877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:46.392985 8877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:46.400220 8883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:46.400228 8884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:46.400466 8887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:46.401590 8877 server_base.cc:1034] running on GCE node
I20260812 14:44:46.401801 8877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:46.402053 8877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:46.403213 8877 hybrid_clock.cc:648] HybridClock initialized: now 1786545886403185 us; error 45 us; skew 500 ppm
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:46.406279 8877 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:44:46.406701 8877 webserver.cc:460] Webserver started at http://127.15.227.195:41633/ using document root <none> and password file <none>
I20260812 14:44:46.406908 8877 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:46.406967 8877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:46.423723 8877 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:46.424422 8902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.424606 8877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:46.424726 8877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "986b70deba3c4522a2c09b35addbaa93"
format_stamp: "Formatted at 2026-08-12 14:44:45 on dist-test-slave-q9h9"
server_key: "ce0e861316f396f24a2033569a5f6a3e"
server_key_iv: "238f6c4c00ca6e9709b7e795159f9d4c"
server_key_version: "kuduclusterkey@0"
I20260812 14:44:46.425223 8877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:46.437263 8877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:46.439355 8877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:46.439496 8877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:46.439731 8877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:46.440069 8877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:46.440115 8877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.440148 8877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:46.440171 8877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:46.446887 8877 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:37141
I20260812 14:44:46.447320 8877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:44:46.447824 9114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:37141 every 8 connection(s)
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:46.455709 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 8877
I20260812 14:44:46.456910 8573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:46.448534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:40075 (local address 127.15.227.254:43061)
0812 14:44:46.448604 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:46.448606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:46.448799 (+ 193us) server_negotiation.cc:407] Connection header received
0812 14:44:46.449396 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:46.449398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:46.449410 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:46.449485 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:46.449994 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.450777 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.451786 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.452072 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.452147 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:46.454140 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:46.454141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:46.454142 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:46.454148 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:46.455498 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.455951 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.455953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.455953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.456016 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.456283 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.456284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.456284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.456416 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:46.456483 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:46.456789 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 14:44:46.456849 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:44:46.457136 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:46.448528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:43061 (local address 127.15.227.195:40075)
0812 14:44:46.448683 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:46.448722 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:46.449282 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:46.449536 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:46.449539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:46.449702 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:46.449873 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:46.449877 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.450883 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.450884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:46.451681 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:46.451684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.452173 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.452175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:46.452271 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:46.452714 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:46.452724 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:46.454057 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:46.455575 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:46.455581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:46.455583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:46.455880 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:46.456092 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:46.456093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:46.456094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:46.456220 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:46.456489 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:46.456495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:46.456645 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 14:44:46.456778 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 14:44:46.456923 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260812 14:44:46.457710 9117 heartbeater.cc:347] Connected to a master server at 127.15.227.254:43061
I20260812 14:44:46.457818 9117 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.457995 9117 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
W20260812 14:44:46.458614 9117 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:43061 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 14:44:46.458683 9117 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.458752 9117 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:46.459081 9117 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:46.459152 9117 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
W20260812 14:44:46.459473 9117 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:44:46.464365 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:46.456072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.15.227.254:43061)
0812 14:44:46.456191 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:46.456193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:46.456344 (+ 151us) server_negotiation.cc:407] Connection header received
0812 14:44:46.456404 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:46.456404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:46.456417 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:46.456477 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:46.458947 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.459629 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.460656 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:46.460833 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:46.460876 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:46.462091 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:46.462092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:46.462093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:46.462100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:46.463460 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.463805 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.463806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.463806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.463851 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:46.463996 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:46.463997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:46.463997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:46.464080 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:46.464190 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:46.464247 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:44:46.464304 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
Aug 12 14:44:46 dist-test-slave-q9h9 krb5kdc[6231](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:47.077340 8568 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:47.077467 8568 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:47.077772 8143 ts_manager.cc:194] Registered new tserver with Master: 74cfa025dce948a88d90bb5ed1503810 (127.15.227.193:42439)
I20260812 14:44:47.078545 8143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:59257
I20260812 14:44:47.272769 8866 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:47.272918 8866 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:47.273213 8143 ts_manager.cc:194] Registered new tserver with Master: 1e47cc55f3dc4cdd856b0d4008301844 (127.15.227.194:45631)
I20260812 14:44:47.273751 8143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:56083
Aug 12 14:44:47 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:47.459656 9117 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:47.459800 9117 heartbeater.cc:511] Master 127.15.227.254:43061 requested a full tablet report, sending...
I20260812 14:44:47.460070 8143 ts_manager.cc:194] Registered new tserver with Master: 986b70deba3c4522a2c09b35addbaa93 (127.15.227.195:37141)
I20260812 14:44:47.460461 8143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:40075
I20260812 14:44:47.467700 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:44:47.480602 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.468687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.15.227.254:43061)
0812 14:44:47.468823 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:47.468825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:47.468854 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:44:47.468911 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:47.468911 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:47.468924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:47.469002 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:47.469663 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.470462 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.471436 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.471744 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.471820 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.477581 (+ 5761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:47.477583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:47.477584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:47.477591 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:47.479374 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:47.479770 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:47.479772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:47.479772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:47.479849 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:47.480049 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:47.480050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:47.480051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:47.480187 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:47.480295 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:47.480415 (+ 120us) server_negotiation.cc:299] Negotiation successful
0812 14:44:47.480504 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260812 14:44:47.482739 8143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34010:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:44:47.485013 8143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:44:47.492954 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.487935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:37141 (local address 127.0.0.1:40738)
0812 14:44:47.488525 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.488564 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:47.488623 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:47.488991 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:47.488995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:47.488997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:47.489070 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.489072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.490443 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.490444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.491675 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.491678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.492663 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.492665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.492759 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.492774 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:44:47.492834 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:44:47.492887 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":106,"threads_started":1}
I20260812 14:44:47.493163 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.488210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.15.227.195:37141)
0812 14:44:47.488834 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:47.488837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:47.488846 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:44:47.488877 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:47.488879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:47.488894 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:47.488969 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:47.489292 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.490358 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.491773 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.492556 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.492663 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.493018 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:47.493047 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 14:44:47.493090 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":63,"threads_started":1}
I20260812 14:44:47.493932 8955 tablet_service.cc:1467] Processing CreateTablet for tablet c00ef2b05f5f4ede856bb5aea78bfbbd (DEFAULT_TABLE table=test-table [id=d516cd4651824ff5836e321a721e576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:47.494371 8955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c00ef2b05f5f4ede856bb5aea78bfbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:47.497397 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.487948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42439 (local address 127.0.0.1:46360)
0812 14:44:47.493470 (+ 5522us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.493480 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:47.493519 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:47.493786 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:47.493787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:47.493789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:47.493819 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.493820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.495208 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.495209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.496252 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.496255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.497196 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.497197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.497252 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.497255 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:44:47.497300 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:44:47.497334 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5512,"thread_start_us":102,"threads_started":1}
I20260812 14:44:47.497743 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.488160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.15.227.193:42439)
0812 14:44:47.488764 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:47.488769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:47.493572 (+ 4803us) server_negotiation.cc:407] Connection header received
0812 14:44:47.493610 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:47.493612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:47.493640 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:47.493706 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:47.493887 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.495119 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.496337 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.497031 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.497115 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.497163 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:47.497615 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 14:44:47.497667 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":59,"threads_started":1}
I20260812 14:44:47.497982 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.488458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45380 (local address 127.15.227.194:45631)
0812 14:44:47.489101 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:47.489105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:47.494221 (+ 5116us) server_negotiation.cc:407] Connection header received
0812 14:44:47.494259 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:47.494262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:47.494277 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:47.494341 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:47.494539 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.495557 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.496758 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.497467 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.497526 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.497574 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:47.497872 (+ 298us) server_negotiation.cc:299] Negotiation successful
0812 14:44:47.497910 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":73,"threads_started":1}
I20260812 14:44:47.497813 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.488368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:45631 (local address 127.0.0.1:45380)
0812 14:44:47.494079 (+ 5711us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.494089 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:47.494159 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:47.494412 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:47.494413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:47.494414 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:47.494469 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.494470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.495642 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.495643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.496674 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.496677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.497608 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.497609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.497676 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.497679 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:44:47.497725 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:44:47.497757 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5693,"thread_start_us":67,"threads_started":1}
I20260812 14:44:47.498605 9178 tablet_bootstrap.cc:492] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Bootstrap starting.
I20260812 14:44:47.498595 8382 tablet_service.cc:1467] Processing CreateTablet for tablet c00ef2b05f5f4ede856bb5aea78bfbbd (DEFAULT_TABLE table=test-table [id=d516cd4651824ff5836e321a721e576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:47.498942 8382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c00ef2b05f5f4ede856bb5aea78bfbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:47.499075 8719 tablet_service.cc:1467] Processing CreateTablet for tablet c00ef2b05f5f4ede856bb5aea78bfbbd (DEFAULT_TABLE table=test-table [id=d516cd4651824ff5836e321a721e576d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:47.499410 8719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c00ef2b05f5f4ede856bb5aea78bfbbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:47.499519 9178 tablet_bootstrap.cc:654] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:47.499886 9178 log.cc:824] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:47.501659 9182 tablet_bootstrap.cc:492] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810: Bootstrap starting.
I20260812 14:44:47.502352 9182 tablet_bootstrap.cc:654] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:47.502725 9182 log.cc:824] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:47.503257 9184 tablet_bootstrap.cc:492] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Bootstrap starting.
I20260812 14:44:47.503491 9182 tablet_bootstrap.cc:492] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810: No bootstrap required, opened a new log
I20260812 14:44:47.503546 9182 ts_tablet_manager.cc:1397] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:47.503870 9184 tablet_bootstrap.cc:654] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:47.504165 9184 log.cc:824] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:47.504889 9184 tablet_bootstrap.cc:492] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: No bootstrap required, opened a new log
I20260812 14:44:47.504943 9184 ts_tablet_manager.cc:1397] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:47.505733 9182 raft_consensus.cc:348] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.505882 9182 raft_consensus.cc:374] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:47.505911 9182 raft_consensus.cc:729] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74cfa025dce948a88d90bb5ed1503810, State: Initialized, Role: FOLLOWER
I20260812 14:44:47.506039 9182 consensus_queue.cc:260] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.506281 9182 ts_tablet_manager.cc:1428] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:44:47.506587 8568 heartbeater.cc:503] Master 127.15.227.254:43061 was elected leader, sending a full tablet report...
I20260812 14:44:47.506979 9184 raft_consensus.cc:348] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.507150 9184 raft_consensus.cc:374] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:47.507186 9184 raft_consensus.cc:729] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e47cc55f3dc4cdd856b0d4008301844, State: Initialized, Role: FOLLOWER
I20260812 14:44:47.507287 9184 consensus_queue.cc:260] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.507531 9184 ts_tablet_manager.cc:1428] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:44:47.507896 8866 heartbeater.cc:503] Master 127.15.227.254:43061 was elected leader, sending a full tablet report...
W20260812 14:44:47.510143 8867 tablet.cc:2369] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:44:47.514559 9178 tablet_bootstrap.cc:492] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: No bootstrap required, opened a new log
I20260812 14:44:47.514639 9178 ts_tablet_manager.cc:1397] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20260812 14:44:47.517046 9178 raft_consensus.cc:348] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.517201 9178 raft_consensus.cc:374] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:47.517233 9178 raft_consensus.cc:729] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 986b70deba3c4522a2c09b35addbaa93, State: Initialized, Role: FOLLOWER
I20260812 14:44:47.517339 9178 consensus_queue.cc:260] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.517572 9178 ts_tablet_manager.cc:1428] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:44:47.517627 9117 heartbeater.cc:503] Master 127.15.227.254:43061 was elected leader, sending a full tablet report...
W20260812 14:44:47.576248 8569 tablet.cc:2369] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 14:44:47 dist-test-slave-q9h9 krb5kdc[6231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for HTTP/127.15.227.212@KRBTEST.COM
I20260812 14:44:47.597918 9192 raft_consensus.cc:482] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:44:47.598100 9192 raft_consensus.cc:504] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.598503 9192 leader_election.cc:288] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74cfa025dce948a88d90bb5ed1503810 (127.15.227.193:42439), 1e47cc55f3dc4cdd856b0d4008301844 (127.15.227.194:45631)
I20260812 14:44:47.605686 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.600975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:42027 (local address 127.15.227.193:42439)
0812 14:44:47.601088 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:47.601091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:47.602070 (+ 979us) server_negotiation.cc:407] Connection header received
0812 14:44:47.602178 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:47.602179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:47.602189 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:47.602217 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:47.602426 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.603157 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.604668 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.605264 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.605311 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.605321 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:47.605556 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 14:44:47.605599 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 14:44:47.605911 9200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.600895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42439 (local address 127.15.227.195:42027)
0812 14:44:47.601978 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.601991 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:47.602102 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:47.602282 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:47.602283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:47.602284 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:47.602360 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.602362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.603263 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.603264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.604548 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.604551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.605402 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.605403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.605475 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.605479 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:44:47.605768 (+ 289us) client_negotiation.cc:240] Negotiation successful
0812 14:44:47.605814 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1062,"thread_start_us":122,"threads_started":1}
I20260812 14:44:47.606237 8465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00ef2b05f5f4ede856bb5aea78bfbbd" candidate_uuid: "986b70deba3c4522a2c09b35addbaa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74cfa025dce948a88d90bb5ed1503810" is_pre_election: true
I20260812 14:44:47.606389 8465 raft_consensus.cc:2454] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 986b70deba3c4522a2c09b35addbaa93 in term 0.
I20260812 14:44:47.606638 8907 leader_election.cc:302] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74cfa025dce948a88d90bb5ed1503810, 986b70deba3c4522a2c09b35addbaa93; no voters:
I20260812 14:44:47.606770 9192 raft_consensus.cc:2781] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:44:47.606808 9192 raft_consensus.cc:482] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:44:47.606828 9192 raft_consensus.cc:3037] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:47.607749 9192 raft_consensus.cc:504] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.607877 9192 leader_election.cc:288] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [CANDIDATE]: Term 1 election: Requested vote from peers 74cfa025dce948a88d90bb5ed1503810 (127.15.227.193:42439), 1e47cc55f3dc4cdd856b0d4008301844 (127.15.227.194:45631)
I20260812 14:44:47.608585 8465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00ef2b05f5f4ede856bb5aea78bfbbd" candidate_uuid: "986b70deba3c4522a2c09b35addbaa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74cfa025dce948a88d90bb5ed1503810"
I20260812 14:44:47.608667 8465 raft_consensus.cc:3037] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:47.608731 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.601496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:35669 (local address 127.15.227.194:45631)
0812 14:44:47.601543 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:47.601546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:47.605477 (+ 3931us) server_negotiation.cc:407] Connection header received
0812 14:44:47.605587 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:47.605588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:47.605598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:47.605628 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:47.605874 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.606547 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.607809 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.608324 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.608367 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.608375 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:47.608621 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 14:44:47.608673 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:44:47.608723 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.601298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:45631 (local address 127.15.227.195:35669)
0812 14:44:47.605407 (+ 4109us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.605418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:47.605510 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:47.605701 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:47.605702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:47.605704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:47.605785 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.605787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.606647 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.606648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.607702 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:47.607718 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.608432 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.608433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:47.608502 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.608507 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:44:47.608589 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 14:44:47.608629 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4087,"thread_start_us":82,"threads_started":1}
I20260812 14:44:47.609072 8763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00ef2b05f5f4ede856bb5aea78bfbbd" candidate_uuid: "986b70deba3c4522a2c09b35addbaa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e47cc55f3dc4cdd856b0d4008301844"
I20260812 14:44:47.609138 8759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c00ef2b05f5f4ede856bb5aea78bfbbd" candidate_uuid: "986b70deba3c4522a2c09b35addbaa93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e47cc55f3dc4cdd856b0d4008301844" is_pre_election: true
I20260812 14:44:47.609200 8763 raft_consensus.cc:3037] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:47.609464 8465 raft_consensus.cc:2454] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 986b70deba3c4522a2c09b35addbaa93 in term 1.
I20260812 14:44:47.609638 8907 leader_election.cc:302] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 74cfa025dce948a88d90bb5ed1503810, 986b70deba3c4522a2c09b35addbaa93; no voters:
I20260812 14:44:47.609773 9192 raft_consensus.cc:2781] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:47.609930 9192 raft_consensus.cc:686] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 1 LEADER]: Becoming Leader. State: Replica: 986b70deba3c4522a2c09b35addbaa93, State: Running, Role: LEADER
I20260812 14:44:47.610040 9192 consensus_queue.cc:237] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } }
I20260812 14:44:47.610121 8763 raft_consensus.cc:2454] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 986b70deba3c4522a2c09b35addbaa93 in term 1.
I20260812 14:44:47.610895 8142 catalog_manager.cc:5495] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 reported cstate change: term changed from 0 to 1, leader changed from <none> to 986b70deba3c4522a2c09b35addbaa93 (127.15.227.195). New cstate: current_term: 1 leader_uuid: "986b70deba3c4522a2c09b35addbaa93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "986b70deba3c4522a2c09b35addbaa93" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 } health_report { overall_health: UNKNOWN } } }
I20260812 14:44:47.642184 9172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.639437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.15.227.195:37141)
0812 14:44:47.639484 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:47.639486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:47.639494 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:44:47.639533 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:47.639534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:47.639555 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:47.639636 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:44:47.639775 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.640501 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.641255 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:47.641554 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:47.641629 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:47.641815 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:44:47.641938 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:44:47.642044 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:47.642086 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 14:44:47.642109 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:44:47.644495 8763 raft_consensus.cc:1261] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 1 FOLLOWER]: Refusing update from remote peer 986b70deba3c4522a2c09b35addbaa93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:47.644590 8465 raft_consensus.cc:1261] T c00ef2b05f5f4ede856bb5aea78bfbbd P 74cfa025dce948a88d90bb5ed1503810 [term 1 FOLLOWER]: Refusing update from remote peer 986b70deba3c4522a2c09b35addbaa93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:47.644871 9192 consensus_queue.cc:1035] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e47cc55f3dc4cdd856b0d4008301844" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:47.645037 9192 consensus_queue.cc:1035] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74cfa025dce948a88d90bb5ed1503810" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:47.647231 9212 mvcc.cc:205] Tried to move back new op lower bound from 7317691955789185024 to 7317691955651334144. Current Snapshot: MvccSnapshot[applied={T|T < 7317691955789185024}]
I20260812 14:44:47.648974 9217 mvcc.cc:205] Tried to move back new op lower bound from 7317691955789185024 to 7317691955651334144. Current Snapshot: MvccSnapshot[applied={T|T < 7317691955789185024}]
I20260812 14:44:47.650328 8143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:47.650417 8143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34010:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:47.651127 8143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ccce921fcb344959c31eb072b04c504: Sending DeleteTablet for 3 replicas of tablet c00ef2b05f5f4ede856bb5aea78bfbbd
I20260812 14:44:47.651523 8955 tablet_service.cc:1514] Processing DeleteTablet for tablet c00ef2b05f5f4ede856bb5aea78bfbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:47 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:40738
I20260812 14:44:47.651804 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 8230
I20260812 14:44:47.651793 8382 tablet_service.cc:1514] Processing DeleteTablet for tablet c00ef2b05f5f4ede856bb5aea78bfbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:47 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:46360
I20260812 14:44:47.654742 8719 tablet_service.cc:1514] Processing DeleteTablet for tablet c00ef2b05f5f4ede856bb5aea78bfbbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:47 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:45380
I20260812 14:44:47.665927 9231 tablet_replica.cc:330] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: stopping tablet replica
I20260812 14:44:47.666126 9231 raft_consensus.cc:2229] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:44:47.666268 9231 pending_rounds.cc:62] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Trying to abort 2 pending ops.
I20260812 14:44:47.666352 9231 pending_rounds.cc:68] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317691955789185024 op_type: WRITE_OP write_request { tablet_id: "c00ef2b05f5f4ede856bb5aea78bfbbd" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\213\220\362\323\006\"8\n\ntest-admin\022*\n d516cd4651824ff5836e321a721e576d\020\001\030\001 \001(\001" signature: "<redacted>""&e\tt\035im\20742r\006)gE\266\001\367\214b\031\254\304\357@\266.~0\251\3615\372GiQx\261~<z\305\017t\215\263\366\177\362\226\252\234\221\204\311~\373f\242u\035U~\253" signing_key_seq_num: 0 } } request_id { client_id: "a123663cbdec462196c74dd2b81ea495" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
W20260812 14:44:47.666463 9222 consensus_queue.cc:1159] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1e47cc55f3dc4cdd856b0d4008301844" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260812 14:44:47.666785 9231 pending_rounds.cc:68] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317691955651334144 op_type: NO_OP noop_request { }
I20260812 14:44:47.666917 9231 raft_consensus.cc:2866] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260812 14:44:47.667317 8908 consensus_peers.cc:487] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 -> Peer 1e47cc55f3dc4cdd856b0d4008301844 (127.15.227.194:45631): Couldn't send request to peer 1e47cc55f3dc4cdd856b0d4008301844. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 14:44:47.667343 9231 raft_consensus.cc:2258] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:47.668502 9229 tablet_replica.cc:330] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: stopping tablet replica
I20260812 14:44:47.668834 9231 ts_tablet_manager.cc:1905] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:47.669517 9229 raft_consensus.cc:2229] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:44:47.669698 9229 raft_consensus.cc:2258] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:47.670943 9231 ts_tablet_manager.cc:1918] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:47.671139 9231 log.cc:1196] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/c00ef2b05f5f4ede856bb5aea78bfbbd
I20260812 14:44:47.671476 9231 ts_tablet_manager.cc:1939] T c00ef2b05f5f4ede856bb5aea78bfbbd P 1e47cc55f3dc4cdd856b0d4008301844: Deleting consensus metadata
I20260812 14:44:47.671500 9229 ts_tablet_manager.cc:1905] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:47.673043 9229 ts_tablet_manager.cc:1918] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:47.673117 9229 log.cc:1196] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/c00ef2b05f5f4ede856bb5aea78bfbbd
I20260812 14:44:47.673378 9229 ts_tablet_manager.cc:1939] T c00ef2b05f5f4ede856bb5aea78bfbbd P 986b70deba3c4522a2c09b35addbaa93: Deleting consensus metadata
I20260812 14:44:47.674326 8107 catalog_manager.cc:4841] TS 986b70deba3c4522a2c09b35addbaa93 (127.15.227.195:37141): tablet c00ef2b05f5f4ede856bb5aea78bfbbd (table test-table [id=d516cd4651824ff5836e321a721e576d]) successfully deleted
I20260812 14:44:47.674436 8111 catalog_manager.cc:4841] TS 1e47cc55f3dc4cdd856b0d4008301844 (127.15.227.194:45631): tablet c00ef2b05f5f4ede856bb5aea78bfbbd (table test-table [id=d516cd4651824ff5836e321a721e576d]) successfully deleted
W20260812 14:44:47.674973 8110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:42439 (error 108)
I20260812 14:44:47.675621 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 8570
I20260812 14:44:47.675849 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.675729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42439 (local address 127.0.0.1:46364)
0812 14:44:47.675778 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.675816 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 14:44:47.675935 8110 catalog_manager.cc:4584] TS 74cfa025dce948a88d90bb5ed1503810 (127.15.227.193:42439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c00ef2b05f5f4ede856bb5aea78bfbbd: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42439: connect: Connection refused (error 111)
I20260812 14:44:47.725225 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 8877
I20260812 14:44:47.728843 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.728725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42439 (local address 127.0.0.1:46366)
0812 14:44:47.728778 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.728804 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:44:47.729327 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:47.729247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42439 (local address 127.0.0.1:46382)
0812 14:44:47.729283 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:47.729303 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 14:44:47.733654 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 8069
I20260812 14:44:47.751904 16271 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 14:44:48.098940 16271 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 14:44:48.099035 16271 mini_ranger.cc:66] Stopping Ranger...
I20260812 14:44:48.442265 16271 mini_ranger.cc:68] Stopped Ranger
2026-08-12 14:44:48.442 UTC [6294] LOG: received smart shutdown request
2026-08-12 14:44:48.446 UTC [6294] LOG: background worker "logical replication launcher" (PID 6305) exited with exit code 1
2026-08-12 14:44:48.446 UTC [6299] LOG: shutting down
2026-08-12 14:44:48.563 UTC [6294] LOG: database system is shut down
2026-08-12T14:44:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45896 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:44:48 dist-test-slave-q9h9 krb5kdc[9251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:44:48 dist-test-slave-q9h9 krb5kdc[9251](info): set up 2 sockets
Aug 12 14:44:48 dist-test-slave-q9h9 krb5kdc[9251](info): commencing operation
krb5kdc: starting...
W20260812 14:44:49.836849 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:44:49 dist-test-slave-q9h9 krb5kdc[9251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:44:49.942296 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:40955
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:40955,127.15.227.253:43629,127.15.227.252:41417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:50.020748 9271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:50.020928 9271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:50.020948 9271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:50.022039 9271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:50.022071 9271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:50.022084 9271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:50.022095 9271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:50.022105 9271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:50.023550 9271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:40955,127.15.227.253:43629,127.15.227.252:41417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:40955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:50.023746 9271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:50.023927 9271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:50.028841 9288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:50.028962 9271 server_base.cc:1034] running on GCE node
W20260812 14:44:50.028932 9284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:50.029058 9285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:50.029333 9271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:50.029613 9271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:50.030776 9271 hybrid_clock.cc:648] HybridClock initialized: now 1786545890030758 us; error 38 us; skew 500 ppm
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:50.033061 9271 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:50.033454 9271 webserver.cc:460] Webserver started at http://127.15.227.254:41987/ using document root <none> and password file <none>
I20260812 14:44:50.033699 9271 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:50.033753 9271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:50.033876 9271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:50.034976 9271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "c95621517c5f4f8381e3288b26fc3a08"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "dffe28459dd4dc996d729a693b5b5607"
server_key_iv: "b479b4bea5eaad80b9ab00a9780f99f8"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.035303 9271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "c95621517c5f4f8381e3288b26fc3a08"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "dffe28459dd4dc996d729a693b5b5607"
server_key_iv: "b479b4bea5eaad80b9ab00a9780f99f8"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.036543 9271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:44:50.037181 9303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:50.037321 9271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:50.037406 9271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "c95621517c5f4f8381e3288b26fc3a08"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "dffe28459dd4dc996d729a693b5b5607"
server_key_iv: "b479b4bea5eaad80b9ab00a9780f99f8"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.037462 9271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:50.059787 9271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:50.061300 9271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:50.061429 9271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:50.065215 9271 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:40955
I20260812 14:44:50.065259 9408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:40955 every 8 connection(s)
I20260812 14:44:50.065547 9271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:50.066265 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 9271
I20260812 14:44:50.066362 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:44:50.066460 9409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:50.066529 16271 external_mini_cluster.cc:1428] Setting key f5d4026fb7fef6b34758b04311717c2d
I20260812 14:44:50.067858 9409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.227.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.253@KRBTEST.COM" created.
I20260812 14:44:50.077313 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.068468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40955 (local address 127.0.0.1:40152)
0812 14:44:50.068630 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.068678 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.069237 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.069472 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.069475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.069777 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.069978 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.069984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.070980 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.070982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.072051 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.072054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.072504 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.072506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.072586 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.073031 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.073042 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.074464 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.076019 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.076025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.076027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.076196 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.076395 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.076395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.076396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.076471 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.076817 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.076824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.076885 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.076958 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.077072 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260812 14:44:50.077322 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.068611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40152 (local address 127.15.227.254:40955)
0812 14:44:50.068715 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.068719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.068750 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:44:50.069355 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.069357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.069372 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.069441 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.070105 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.070891 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.072128 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.072419 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.072486 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.074537 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.074543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.074543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.074550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.075949 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.076268 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.076270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.076270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.076339 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.076524 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.076524 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.076525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.076734 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.076806 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.076991 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.077141 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":74,"thread_start_us":36,"threads_started":1}
I20260812 14:44:50.078073 9409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:44:50.078353 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.078254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56652)
0812 14:44:50.078294 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.078335 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:44:50.078473 9309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
I20260812 14:44:50.078984 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.078889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56654)
0812 14:44:50.078939 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.078965 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:44:50.079082 9409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:43629: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab.
Entry for principal kudu/127.15.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab.
I20260812 14:44:50.089866 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:43629
--webserver_interface=127.15.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab
--principal=kudu/127.15.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:40955,127.15.227.253:43629,127.15.227.252:41417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:50.128285 9409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } attempt: 1
I20260812 14:44:50.129381 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.129252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56668)
0812 14:44:50.129312 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.129352 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 14:44:50.134869 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.134762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56676)
0812 14:44:50.134818 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.134842 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260812 14:44:50.134943 9409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:43629: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
W20260812 14:44:50.170202 9418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:50.170372 9418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:50.170403 9418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:50.171489 9418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:50.171526 9418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:50.171545 9418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:50.171561 9418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:50.171586 9418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:50.173008 9418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:40955,127.15.227.253:43629,127.15.227.252:41417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab
--principal=kudu/127.15.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:43629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:50.173240 9418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:50.173424 9418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:50.177843 9429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:50.177963 9418 server_base.cc:1034] running on GCE node
W20260812 14:44:50.178056 9431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:50.178062 9433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:50.178304 9418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:50.178565 9418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:50.179705 9418 hybrid_clock.cc:648] HybridClock initialized: now 1786545890179692 us; error 27 us; skew 500 ppm
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:50.181892 9418 init.cc:377] Logged in from keytab as kudu/127.15.227.253@KRBTEST.COM (short username kudu)
I20260812 14:44:50.182337 9418 webserver.cc:460] Webserver started at http://127.15.227.253:36685/ using document root <none> and password file <none>
I20260812 14:44:50.182520 9418 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:50.182566 9418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:50.182682 9418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:50.183635 9418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data/instance:
uuid: "36d605ec8a1a42038b3ae2cfa1c44448"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "0fba1ce268fb3a6285f6afca6dfe121d"
server_key_iv: "bf57d5ea542d91020e219f5cf64a5a7b"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.183946 9418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/wal/instance:
uuid: "36d605ec8a1a42038b3ae2cfa1c44448"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "0fba1ce268fb3a6285f6afca6dfe121d"
server_key_iv: "bf57d5ea542d91020e219f5cf64a5a7b"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.185122 9418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:44:50.185774 9447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:50.185926 9418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:50.186030 9418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/wal
uuid: "36d605ec8a1a42038b3ae2cfa1c44448"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "0fba1ce268fb3a6285f6afca6dfe121d"
server_key_iv: "bf57d5ea542d91020e219f5cf64a5a7b"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.186101 9418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:50.198724 9418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:50.200177 9418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:50.200284 9418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:50.203069 9409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } attempt: 2
I20260812 14:44:50.203433 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.203308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56678)
0812 14:44:50.203366 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.203403 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 14:44:50.203759 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.203663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56688)
0812 14:44:50.203707 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.203733 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:44:50.203845 9409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:43629: Network error: Client connection negotiation failed: client connection to 127.15.227.253:43629: connect: Connection refused (error 111)
I20260812 14:44:50.204140 9418 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.253:43629
I20260812 14:44:50.204175 9555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.253:43629 every 8 connection(s)
I20260812 14:44:50.204465 9418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
I20260812 14:44:50.205192 9559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:50.205449 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 9418
I20260812 14:44:50.205546 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-1/wal/instance
I20260812 14:44:50.205720 16271 external_mini_cluster.cc:1428] Setting key 259036c842d11048afdc85e047d43837
I20260812 14:44:50.206686 9559 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.227.252@KRBTEST.COM; defaulting to no policy
I20260812 14:44:50.215440 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.207237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40164 (local address 127.15.227.254:40955)
0812 14:44:50.207282 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.207284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.207516 (+ 232us) server_negotiation.cc:407] Connection header received
0812 14:44:50.208071 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.208072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.208084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.208121 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.208832 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.209303 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.210751 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.210953 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.210998 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.213120 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.213122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.213122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.213128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.214393 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.214753 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.214754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.214754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.214823 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.215000 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.215000 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.215000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.215091 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.215134 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.215323 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.215387 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:44:50.215566 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.207233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40955 (local address 127.0.0.1:40164)
0812 14:44:50.207387 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.207432 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.207972 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.208188 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.208192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.208499 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.208757 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.208763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.209386 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.209387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.210657 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.210660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.211023 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.211024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.211167 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.211570 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.211580 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.213037 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.214460 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.214467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.214469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.214675 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.214872 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.214872 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.214872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.214959 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.215130 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.215133 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.215233 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.215305 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.215408 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260812 14:44:50.216058 9559 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.252@KRBTEST.COM" created.
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:44:50.222275 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.216331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.15.227.253:43629)
0812 14:44:50.216480 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.216484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.216500 (+ 16us) server_negotiation.cc:407] Connection header received
0812 14:44:50.216521 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.216523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.216535 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.216608 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.216874 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.217513 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.218153 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.218432 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.218488 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.219957 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.219964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.219964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.219970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.221244 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.221521 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.221524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.221524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.221573 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.221723 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.221724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.221724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.221852 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.221917 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.222047 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.222151 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260812 14:44:50.222326 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.216279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56694)
0812 14:44:50.216316 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.216324 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.216359 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.216611 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.216612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.216758 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.216810 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.216812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.217594 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.217595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.218094 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.218096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.218494 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.218494 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.218551 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.218884 (+ 333us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.218889 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.219888 (+ 999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.221315 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.221316 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.221316 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.221460 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.221624 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.221625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.221625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.221676 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.221908 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.221909 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.221944 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.222003 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.222199 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":85}
I20260812 14:44:50.223001 9559 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:44:50.223278 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.223168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41196)
0812 14:44:50.223205 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.223258 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 14:44:50.223387 9456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
I20260812 14:44:50.223743 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.223660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41204)
0812 14:44:50.223699 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.223722 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:44:50.223829 9559 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:41417: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab.
Entry for principal kudu/127.15.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab.
I20260812 14:44:50.228523 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:41417
--webserver_interface=127.15.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab
--principal=kudu/127.15.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:40955,127.15.227.253:43629,127.15.227.252:41417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:50.273005 9559 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } attempt: 1
I20260812 14:44:50.273384 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.273280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41216)
0812 14:44:50.273331 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.273362 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:44:50.273672 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.273597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41232)
0812 14:44:50.273634 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.273657 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:44:50.273769 9559 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:41417: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
I20260812 14:44:50.294966 9409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } attempt: 3
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:44:50.301669 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.295260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56708)
0812 14:44:50.295306 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.295317 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.295372 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.295498 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.295499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.295633 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.295699 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.295700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.296611 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.296612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.297118 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.297119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.297568 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.297568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.297618 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.298046 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.298050 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.299162 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.300771 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.300772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.300772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.300943 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.301123 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.301124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.301124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.301173 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.301381 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.301382 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.301427 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.301480 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.301572 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260812 14:44:50.301669 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.295318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56708 (local address 127.15.227.253:43629)
0812 14:44:50.295378 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.295380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.295391 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:44:50.295416 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.295417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.295429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.295484 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.295764 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.296541 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.297194 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.297502 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.297575 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.299241 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.299243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.299243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.299250 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.300696 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.301014 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.301016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.301016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.301069 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.301222 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.301224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.301224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.301326 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.301383 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.301510 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.301588 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 14:44:50.302130 9409 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:44:50.302412 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.302331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41244)
0812 14:44:50.302364 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.302389 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260812 14:44:50.302676 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.302594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41252)
0812 14:44:50.302632 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.302656 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 14:44:50.302783 9409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:41417: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
W20260812 14:44:50.309253 9573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:50.309415 9573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:50.309437 9573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:50.310518 9573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:50.310554 9573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:50.310566 9573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:50.310582 9573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:50.310593 9573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:50.311960 9573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:40955,127.15.227.253:43629,127.15.227.252:41417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab
--principal=kudu/127.15.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:41417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:50.312139 9573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:50.312305 9573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:50.316645 9587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:50.316726 9573 server_base.cc:1034] running on GCE node
W20260812 14:44:50.316694 9590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:50.316648 9583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:50.317092 9573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:50.317349 9573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:50.318485 9573 hybrid_clock.cc:648] HybridClock initialized: now 1786545890318469 us; error 25 us; skew 500 ppm
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:50.320570 9573 init.cc:377] Logged in from keytab as kudu/127.15.227.252@KRBTEST.COM (short username kudu)
I20260812 14:44:50.320971 9573 webserver.cc:460] Webserver started at http://127.15.227.252:44619/ using document root <none> and password file <none>
I20260812 14:44:50.321190 9573 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:50.321237 9573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:50.321359 9573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:50.322331 9573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data/instance:
uuid: "0701daa940c54082ac0c0d7b798eef0a"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "053b7dcbcaba23eb857688947f05e82a"
server_key_iv: "38f7505dd068de872ee1a0493c420235"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.322641 9573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/wal/instance:
uuid: "0701daa940c54082ac0c0d7b798eef0a"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "053b7dcbcaba23eb857688947f05e82a"
server_key_iv: "38f7505dd068de872ee1a0493c420235"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.323798 9573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:44:50.324424 9605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:50.324568 9573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:50.324645 9573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/wal
uuid: "0701daa940c54082ac0c0d7b798eef0a"
format_stamp: "Formatted at 2026-08-12 14:44:50 on dist-test-slave-q9h9"
server_key: "053b7dcbcaba23eb857688947f05e82a"
server_key_iv: "38f7505dd068de872ee1a0493c420235"
server_key_version: "encryptionkey@0"
I20260812 14:44:50.324707 9573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:50.333899 9409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } attempt: 1
I20260812 14:44:50.334218 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.334106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41260)
0812 14:44:50.334156 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.334189 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:44:50.334483 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.334411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41266)
0812 14:44:50.334446 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.334468 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 14:44:50.334565 9409 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:41417: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
I20260812 14:44:50.341912 9559 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } attempt: 2
I20260812 14:44:50.342252 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.342154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41276)
0812 14:44:50.342198 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.342225 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:44:50.342555 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.342460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41292)
0812 14:44:50.342501 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.342532 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 14:44:50.342630 9559 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:41417: Network error: Client connection negotiation failed: client connection to 127.15.227.252:41417: connect: Connection refused (error 111)
I20260812 14:44:50.348834 9573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:50.350383 9573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:50.350483 9573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:50.354081 9573 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.252:41417
I20260812 14:44:50.354115 9716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.252:41417 every 8 connection(s)
I20260812 14:44:50.354413 9573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
I20260812 14:44:50.355188 9717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:50.356564 9717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:50.363080 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 9573
I20260812 14:44:50.363209 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786545740023890-16271-0/minicluster-data/master-2/wal/instance
I20260812 14:44:50.363402 16271 external_mini_cluster.cc:1428] Setting key 2f1157e1e09009c1af5ca2be552fc200
I20260812 14:44:50.364212 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.357145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.15.227.254:40955)
0812 14:44:50.357210 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.357212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.357376 (+ 164us) server_negotiation.cc:407] Connection header received
0812 14:44:50.357787 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.357788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.357797 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.357840 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.358391 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.358859 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.359715 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.359885 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.359925 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.361691 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.361693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.361693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.361699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.362767 (+ 1068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.363185 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.363186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.363187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.363239 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.363520 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.363521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.363522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.363660 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.363732 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.364049 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.364129 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 14:44:50.364355 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.357137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40955 (local address 127.0.0.1:40166)
0812 14:44:50.357278 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.357309 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.357705 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.357893 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.357896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.358146 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.358327 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.358331 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.358917 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.358918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.359657 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.359658 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.359932 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.359933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.360028 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.360422 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.360430 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.361615 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.362835 (+ 1220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.362845 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.362847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.363117 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.363302 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.363303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.363303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.363430 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.363735 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.363743 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.363897 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.364004 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.364175 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260812 14:44:50.364856 9717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:44:50.370579 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.365059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:43629 (local address 127.0.0.1:56720)
0812 14:44:50.365103 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.365114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.365173 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.365286 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.365286 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.365426 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.365470 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.365471 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.366080 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.366081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.366545 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.366546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.366855 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.366856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.366912 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.367273 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.367277 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.368320 (+ 1043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.369686 (+ 1366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.369686 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.369687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.369865 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.370046 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.370047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.370047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.370102 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.370313 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.370313 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.370354 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.370414 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.370496 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:44:50.370594 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.365113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.15.227.253:43629)
0812 14:44:50.365151 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.365152 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.365160 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:44:50.365232 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.365233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.365240 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.365280 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.365527 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.366004 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.366620 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.366799 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.366845 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.368392 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.368393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.368393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.368400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.369615 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.369924 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.369925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.369925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.369979 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.370152 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.370154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.370154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.370260 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.370313 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.370442 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.370513 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:44:50.370962 9717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:44:50.371512 9721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.363721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.15.227.252:41417)
0812 14:44:50.363852 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.363857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.363978 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:44:50.364009 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.364012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.364026 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.364097 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.364968 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.365834 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.366603 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.366925 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.366992 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.368945 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.368952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.368952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.368958 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.370364 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.370684 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.370688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.370688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.370752 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.370953 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.370954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.370954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.371136 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.371269 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.371350 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.371418 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:50.378005 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.371189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41310 (local address 127.15.227.252:41417)
0812 14:44:50.371303 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.371305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.371314 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:44:50.371365 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.371365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.371375 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.371430 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.371648 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.372156 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.372968 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.373286 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.373344 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.374944 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.374946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.374946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.374952 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.377047 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.377375 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.377376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.377376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.377426 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.377596 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.377596 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.377596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.377674 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.377717 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.377858 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.377916 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
I20260812 14:44:50.378113 9720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.371161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41310)
0812 14:44:50.371280 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.371289 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.371325 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.371429 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.371430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.371545 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.371573 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.371574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.372220 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.372221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.372904 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.372906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.373352 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.373352 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.373384 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.373667 (+ 283us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.373671 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.374873 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.377117 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.377118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.377119 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.377299 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.377480 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.377481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.377481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.377540 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.377725 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.377726 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.377769 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.377843 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.377976 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":51}
I20260812 14:44:50.379223 9717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: Bootstrap starting.
I20260812 14:44:50.379799 9717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:50.380054 9717 log.cc:824] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:50.380712 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.372155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40182 (local address 127.15.227.254:40955)
0812 14:44:50.372191 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.372193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.372200 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:44:50.372219 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.372220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.372229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.372276 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.372797 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.373420 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.374278 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.374562 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.374635 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.376522 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.376524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.376524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.376530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.377896 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.378245 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.378246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.378246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.378299 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.380267 (+ 1968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.380268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.380268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.380370 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.380525 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.380566 (+ 41us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.380634 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:44:50.380821 9717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: No bootstrap required, opened a new log
I20260812 14:44:50.382535 9717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.382692 9717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:50.382735 9717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0701daa940c54082ac0c0d7b798eef0a, State: Initialized, Role: FOLLOWER
I20260812 14:44:50.382846 9717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.383122 9717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:50.383083 9730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.383188 9730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:44:50.384984 9757 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:44:50.385075 9757 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:44:50.385587 9573 master_runner.cc:186] Error getting master registration for 127.15.227.254:40955: OK, instance_id { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" instance_seqno: 1786545890064215 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:44:50.387666 9573 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:44:50.388146 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.381279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.15.227.253:43629)
0812 14:44:50.381310 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.381312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.381318 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:44:50.381388 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.381389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.381396 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.381434 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.381955 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.382406 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.383261 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.383535 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.383579 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.385861 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.385862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.385862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.385869 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.387044 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.387334 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.387335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.387336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.387388 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.387545 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.387546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.387546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.387660 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.387814 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.387987 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.388063 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:44:50.389488 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:44:50.398202 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.390861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.15.227.254:40955)
0812 14:44:50.390946 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.390948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.391318 (+ 370us) server_negotiation.cc:407] Connection header received
0812 14:44:50.391404 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.391404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.391414 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.391439 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.392226 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.392866 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.393755 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.394005 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.394076 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.395389 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.395390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.395391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.395397 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.396997 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.397364 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.397365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.397365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.397420 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.397613 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.397613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.397614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.397741 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.397818 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.398049 (+ 231us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.398124 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 14:44:50.398339 9565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.390905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.15.227.253:43629)
0812 14:44:50.390940 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.390942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.391157 (+ 215us) server_negotiation.cc:407] Connection header received
0812 14:44:50.391251 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.391252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.391262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.391292 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.391940 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.392592 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.393543 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.393814 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.393849 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.395389 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.395390 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.395390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.395397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.397120 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.397509 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.397510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.397510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.397566 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.397769 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.397770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.397770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.397885 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.397949 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.398176 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.398258 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:44:50.399242 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.391230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41320 (local address 127.15.227.252:41417)
0812 14:44:50.391269 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.391270 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.391666 (+ 396us) server_negotiation.cc:407] Connection header received
0812 14:44:50.391724 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.391725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.391745 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.391793 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.392470 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.393151 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.394189 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.394511 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.394550 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.395924 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.395926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.395926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.395933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.397310 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.397633 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.397633 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.397634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.397681 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.397881 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.397882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.397883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.398057 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.398135 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.399099 (+ 964us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.399168 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:44:50.409705 9409 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } attempt: 2
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:44:50.415818 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.409947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41328)
0812 14:44:50.409993 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.410003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.410058 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.410185 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.410186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.410294 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.410323 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.410324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.411154 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.411155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.411695 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.411698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.412102 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.412103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.412137 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.412485 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.412489 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.413481 (+ 992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.414992 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.414992 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.414993 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.415196 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.415366 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.415366 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.415367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.415416 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.415624 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.415624 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.415655 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.415699 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.415763 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 14:44:50.415871 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.410054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41328 (local address 127.15.227.252:41417)
0812 14:44:50.410086 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.410088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.410095 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:44:50.410112 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.410112 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.410121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.410176 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.410378 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.411077 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.411762 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.412040 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.412106 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.413576 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.413578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.413578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.413585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.414925 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.415253 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.415254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.415254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.415308 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.415462 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.415463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.415463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.415573 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.415633 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.415721 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.415790 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:44:50.417207 9409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: Bootstrap starting.
I20260812 14:44:50.417886 9409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:50.418219 9409 log.cc:824] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:50.418960 9409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: No bootstrap required, opened a new log
I20260812 14:44:50.420867 9409 raft_consensus.cc:348] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.420995 9409 raft_consensus.cc:374] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:50.421036 9409 raft_consensus.cc:729] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95621517c5f4f8381e3288b26fc3a08, State: Initialized, Role: FOLLOWER
I20260812 14:44:50.421144 9409 consensus_queue.cc:260] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.421432 9773 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.421485 9409 sys_catalog.cc:564] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:50.421507 9773 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:44:50.423272 9795 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:44:50.423329 9795 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:44:50.423766 9271 master_runner.cc:186] Error getting master registration for 127.15.227.253:43629: OK, instance_id { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" instance_seqno: 1786545890202936 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:44:50.425961 9271 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:44:50.433764 9559 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } attempt: 3
Aug 12 14:44:50 dist-test-slave-q9h9 krb5kdc[9251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545890, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:44:50.441375 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.434056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.15.227.252:41417)
0812 14:44:50.434087 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:50.434089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:50.434099 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:44:50.434157 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:50.434158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:50.434166 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:50.434223 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:50.434500 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.435201 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.436118 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.436412 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.436475 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.438426 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:50.438428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:50.438428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:50.438435 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:50.440212 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.440589 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.440591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.440591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.440647 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:50.440848 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:50.440849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:50.440849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:50.441010 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:50.441075 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:50.441218 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 14:44:50.441290 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:44:50.441380 9564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:50.433992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:41417 (local address 127.0.0.1:41332)
0812 14:44:50.434061 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:50.434071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:50.434111 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:50.434233 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:50.434233 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:50.434392 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:50.434434 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.434436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.435276 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.435277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.436042 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:50.436044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:50.436479 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:50.436480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:50.436534 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:50.436931 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:50.436936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:50.438340 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:50.440299 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.440300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.440300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.440505 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.440714 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:50.440715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:50.440715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:50.440786 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:50.441072 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:50.441073 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:50.441118 (+ 45us) client_negotiation.cc:769] Sending connection context
0812 14:44:50.441185 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 14:44:50.441288 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 14:44:50.443205 9559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448: Bootstrap starting.
I20260812 14:44:50.443881 9559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:50.444181 9559 log.cc:824] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:50.445003 9559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448: No bootstrap required, opened a new log
I20260812 14:44:50.446899 9559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.447028 9559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:50.447062 9559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d605ec8a1a42038b3ae2cfa1c44448, State: Initialized, Role: FOLLOWER
I20260812 14:44:50.447171 9559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.447466 9559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:50.447440 9801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.447614 9801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:44:50.449430 9822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:44:50.449541 9822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 14:44:50.450769 9459 tablet.cc:2369] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:44:50.451311 9418 master_runner.cc:263] No leader master found from master 36d605ec8a1a42038b3ae2cfa1c44448
I20260812 14:44:50.451359 9418 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 14:44:50.562098 9315 tablet.cc:2369] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 14:44:50.600970 9617 tablet.cc:2369] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:44:50.811323 9801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:44:50.811434 9801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.811678 9801 leader_election.cc:288] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c95621517c5f4f8381e3288b26fc3a08 (127.15.227.254:40955), 0701daa940c54082ac0c0d7b798eef0a (127.15.227.252:41417)
I20260812 14:44:50.811937 9664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0701daa940c54082ac0c0d7b798eef0a" is_pre_election: true
I20260812 14:44:50.811937 9351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95621517c5f4f8381e3288b26fc3a08" is_pre_election: true
I20260812 14:44:50.812103 9664 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d605ec8a1a42038b3ae2cfa1c44448 in term 0.
I20260812 14:44:50.812101 9351 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36d605ec8a1a42038b3ae2cfa1c44448 in term 0.
I20260812 14:44:50.812295 9456 leader_election.cc:302] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0701daa940c54082ac0c0d7b798eef0a, 36d605ec8a1a42038b3ae2cfa1c44448; no voters:
I20260812 14:44:50.812448 9801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:44:50.812510 9801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:44:50.812532 9801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:50.813256 9801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.813361 9801 leader_election.cc:288] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [CANDIDATE]: Term 1 election: Requested vote from peers c95621517c5f4f8381e3288b26fc3a08 (127.15.227.254:40955), 0701daa940c54082ac0c0d7b798eef0a (127.15.227.252:41417)
I20260812 14:44:50.813519 9351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c95621517c5f4f8381e3288b26fc3a08"
I20260812 14:44:50.813534 9664 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0701daa940c54082ac0c0d7b798eef0a"
I20260812 14:44:50.813607 9351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:50.813607 9664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:50.814509 9664 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d605ec8a1a42038b3ae2cfa1c44448 in term 1.
I20260812 14:44:50.814509 9351 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36d605ec8a1a42038b3ae2cfa1c44448 in term 1.
I20260812 14:44:50.814700 9456 leader_election.cc:302] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36d605ec8a1a42038b3ae2cfa1c44448, c95621517c5f4f8381e3288b26fc3a08; no voters:
I20260812 14:44:50.814810 9801 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:50.814944 9801 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [term 1 LEADER]: Becoming Leader. State: Replica: 36d605ec8a1a42038b3ae2cfa1c44448, State: Running, Role: LEADER
I20260812 14:44:50.815030 9801 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } }
I20260812 14:44:50.815408 9828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36d605ec8a1a42038b3ae2cfa1c44448. Latest consensus state: current_term: 1 leader_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.815479 9828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:50.815673 9834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:50.815840 9834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:44:50.817121 9351 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [term 1 FOLLOWER]: Refusing update from remote peer 36d605ec8a1a42038b3ae2cfa1c44448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:50.817139 9664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [term 1 FOLLOWER]: Refusing update from remote peer 36d605ec8a1a42038b3ae2cfa1c44448: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:50.817427 9828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:50.817539 9801 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:50.818714 9773 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36d605ec8a1a42038b3ae2cfa1c44448. Latest consensus state: current_term: 1 leader_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.818810 9773 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:44:50.818768 9730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36d605ec8a1a42038b3ae2cfa1c44448. Latest consensus state: current_term: 1 leader_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.818842 9730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:44:50.819576 9828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.819650 9828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:50.819804 9838 mvcc.cc:205] Tried to move back new op lower bound from 7317691968784830464 to 7317691968778928128. Current Snapshot: MvccSnapshot[applied={T|T < 7317691968784830464}]
I20260812 14:44:50.819996 9801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.820047 9834 catalog_manager.cc:1347] Generated new cluster ID: aa9ccd1ab321408182719821971260fd
I20260812 14:44:50.820065 9801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448 [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:50.820116 9834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:44:50.820418 9773 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.820497 9773 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:44:50.820648 9730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c95621517c5f4f8381e3288b26fc3a08" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40955 } } peers { permanent_uuid: "36d605ec8a1a42038b3ae2cfa1c44448" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 43629 } } peers { permanent_uuid: "0701daa940c54082ac0c0d7b798eef0a" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 41417 } } }
I20260812 14:44:50.820725 9730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:44:50.833670 9834 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:44:50.834285 9834 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:44:50.839740 9834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36d605ec8a1a42038b3ae2cfa1c44448: Generated new TSK 0
I20260812 14:44:50.839993 9834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:44:51.385520 9757 catalog_manager.cc:1259] Loaded cluster ID: aa9ccd1ab321408182719821971260fd
I20260812 14:44:51.385589 9757 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: loading cluster ID for follower catalog manager: success
I20260812 14:44:51.386415 9757 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: acquiring CA information for follower catalog manager: success
I20260812 14:44:51.386708 9757 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0701daa940c54082ac0c0d7b798eef0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:44:51.423698 9795 catalog_manager.cc:1259] Loaded cluster ID: aa9ccd1ab321408182719821971260fd
I20260812 14:44:51.423755 9795 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: loading cluster ID for follower catalog manager: success
I20260812 14:44:51.424504 9795 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: acquiring CA information for follower catalog manager: success
I20260812 14:44:51.424748 9795 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c95621517c5f4f8381e3288b26fc3a08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:44:51.903194 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 9271
I20260812 14:44:51.921063 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 9418
I20260812 14:44:51.932353 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 9573
2026-08-12T14:44:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3278 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:44:51 dist-test-slave-q9h9 krb5kdc[9859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:44:51 dist-test-slave-q9h9 krb5kdc[9859](info): set up 2 sockets
Aug 12 14:44:51 dist-test-slave-q9h9 krb5kdc[9859](info): commencing operation
krb5kdc: starting...
W20260812 14:44:53.128080 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:44:53 dist-test-slave-q9h9 krb5kdc[9859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:44:53.234820 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45883
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:45883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:53.312255 9893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:53.312436 9893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:53.312464 9893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:53.313555 9893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:53.313593 9893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:53.313611 9893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:53.313627 9893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:53.313643 9893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:53.315091 9893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:45883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:53.315330 9893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:53.315522 9893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:53.319814 9902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:53.319841 9906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:53.319977 9893 server_base.cc:1034] running on GCE node
W20260812 14:44:53.319851 9912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:53.320291 9893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:53.320575 9893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:53.321735 9893 hybrid_clock.cc:648] HybridClock initialized: now 1786545893321714 us; error 33 us; skew 500 ppm
Aug 12 14:44:53 dist-test-slave-q9h9 krb5kdc[9859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545893, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:53.329491 9893 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:53.329900 9893 webserver.cc:460] Webserver started at http://127.15.227.254:42171/ using document root <none> and password file <none>
I20260812 14:44:53.330139 9893 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:53.330190 9893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:53.330310 9893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:53.331252 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "7c5171e504e44d699a966a02a97fdcef"
format_stamp: "Formatted at 2026-08-12 14:44:53 on dist-test-slave-q9h9"
server_key: "73aa56939b4d5c5ab8c1bdf82e1faf36"
server_key_iv: "784f61ea3243f310b7ffb5886ea42c6a"
server_key_version: "encryptionkey@0"
I20260812 14:44:53.331562 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "7c5171e504e44d699a966a02a97fdcef"
format_stamp: "Formatted at 2026-08-12 14:44:53 on dist-test-slave-q9h9"
server_key: "73aa56939b4d5c5ab8c1bdf82e1faf36"
server_key_iv: "784f61ea3243f310b7ffb5886ea42c6a"
server_key_version: "encryptionkey@0"
I20260812 14:44:53.332756 9893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:44:53.333388 9921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:53.333537 9893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:53.333618 9893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "7c5171e504e44d699a966a02a97fdcef"
format_stamp: "Formatted at 2026-08-12 14:44:53 on dist-test-slave-q9h9"
server_key: "73aa56939b4d5c5ab8c1bdf82e1faf36"
server_key_iv: "784f61ea3243f310b7ffb5886ea42c6a"
server_key_version: "encryptionkey@0"
I20260812 14:44:53.333678 9893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:53.359532 9893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:53.361107 9893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:53.361251 9893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:53.364979 9893 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:45883
I20260812 14:44:53.365015 10043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:45883 every 8 connection(s)
I20260812 14:44:53.365309 9893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:53.366184 10044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:53.368275 10044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef: Bootstrap starting.
I20260812 14:44:53.368752 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 9893
I20260812 14:44:53.368818 10044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:53.368847 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:44:53.369024 16271 external_mini_cluster.cc:1428] Setting key 59807cb9b167767092eb97d20435851c
I20260812 14:44:53.369077 10044 log.cc:824] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:53.369836 10044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef: No bootstrap required, opened a new log
I20260812 14:44:53.371953 10044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5171e504e44d699a966a02a97fdcef" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45883 } }
I20260812 14:44:53.372120 10044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:53.372166 10044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c5171e504e44d699a966a02a97fdcef, State: Initialized, Role: FOLLOWER
I20260812 14:44:53.372288 10044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5171e504e44d699a966a02a97fdcef" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45883 } }
I20260812 14:44:53.372368 10044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:44:53.372401 10044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:44:53.372445 10044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:53.373060 10044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5171e504e44d699a966a02a97fdcef" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45883 } }
I20260812 14:44:53.373148 10044 leader_election.cc:302] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c5171e504e44d699a966a02a97fdcef; no voters:
I20260812 14:44:53.373319 10044 leader_election.cc:288] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:44:53.373355 10055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:53.373517 10055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [term 1 LEADER]: Becoming Leader. State: Replica: 7c5171e504e44d699a966a02a97fdcef, State: Running, Role: LEADER
I20260812 14:44:53.373569 10044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:53.373682 10055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5171e504e44d699a966a02a97fdcef" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45883 } }
I20260812 14:44:53.374047 10055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c5171e504e44d699a966a02a97fdcef. Latest consensus state: current_term: 1 leader_uuid: "7c5171e504e44d699a966a02a97fdcef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5171e504e44d699a966a02a97fdcef" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45883 } } }
I20260812 14:44:53.374262 10055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:53.374194 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c5171e504e44d699a966a02a97fdcef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c5171e504e44d699a966a02a97fdcef" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45883 } } }
I20260812 14:44:53.374660 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:53.374812 10070 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:53.375319 10070 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:44:53 dist-test-slave-q9h9 krb5kdc[9859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:53.376750 10070 catalog_manager.cc:1347] Generated new cluster ID: 96fba0ac2f854fa485412fbc020f5060
I20260812 14:44:53.376799 10070 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:44:53.380905 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:53.369401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46434 (local address 127.15.227.254:45883)
0812 14:44:53.369550 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:53.369560 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:53.369589 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:44:53.370097 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:53.370104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:53.370170 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:53.370274 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:53.371057 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:53.371894 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:53.372918 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:53.373367 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:53.373755 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:53.376282 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:53.376286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:53.376286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:53.376297 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:53.379441 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:53.379824 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:53.379827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:53.379827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:53.379921 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:53.380109 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:53.380110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:53.380110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:53.380369 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:53.380517 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:53.380609 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 14:44:53.380729 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260812 14:44:53.381992 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:44:53.387639 10070 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:44:53.388294 10070 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:44:53.393150 10070 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c5171e504e44d699a966a02a97fdcef: Generated new TSK 0
I20260812 14:44:53.393297 10070 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:44:53.435545 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 9893
2026-08-12T14:44:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1502 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:44:53 dist-test-slave-q9h9 krb5kdc[10099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:44:53 dist-test-slave-q9h9 krb5kdc[10099](info): set up 2 sockets
Aug 12 14:44:53 dist-test-slave-q9h9 krb5kdc[10099](info): commencing operation
krb5kdc: starting...
W20260812 14:44:54.620041 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:44:54 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:44:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:44:54.725818 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45731
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:45731
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:54.802785 10128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:54.802958 10128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:54.802976 10128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:54.804075 10128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:54.804111 10128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:54.804124 10128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:54.804147 10128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:54.804163 10128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:54.805573 10128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:45731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:54.805787 10128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:54.805979 10128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:54.810334 10140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:54.810314 10142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:54.810426 10128 server_base.cc:1034] running on GCE node
W20260812 14:44:54.810317 10147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:54.810822 10128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:54.811069 10128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:54.812206 10128 hybrid_clock.cc:648] HybridClock initialized: now 1786545894812195 us; error 29 us; skew 500 ppm
Aug 12 14:44:54 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545894, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:54.814517 10128 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:54.814945 10128 webserver.cc:460] Webserver started at http://127.15.227.254:37915/ using document root <none> and password file <none>
I20260812 14:44:54.815135 10128 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:54.815179 10128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:54.815300 10128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:54.816220 10128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "7e0b6218bf604053bff2ac2006057e7c"
format_stamp: "Formatted at 2026-08-12 14:44:54 on dist-test-slave-q9h9"
server_key: "466c20f7bf64b63013edb3c383fd02f5"
server_key_iv: "8252f38dbc75371df4d1715f08dbff98"
server_key_version: "encryptionkey@0"
I20260812 14:44:54.816529 10128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "7e0b6218bf604053bff2ac2006057e7c"
format_stamp: "Formatted at 2026-08-12 14:44:54 on dist-test-slave-q9h9"
server_key: "466c20f7bf64b63013edb3c383fd02f5"
server_key_iv: "8252f38dbc75371df4d1715f08dbff98"
server_key_version: "encryptionkey@0"
I20260812 14:44:54.817673 10128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:44:54.818385 10162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:54.818531 10128 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:54.818621 10128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "7e0b6218bf604053bff2ac2006057e7c"
format_stamp: "Formatted at 2026-08-12 14:44:54 on dist-test-slave-q9h9"
server_key: "466c20f7bf64b63013edb3c383fd02f5"
server_key_iv: "8252f38dbc75371df4d1715f08dbff98"
server_key_version: "encryptionkey@0"
I20260812 14:44:54.818681 10128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:54.849181 10128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:54.851075 10128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:54.851200 10128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:54.854883 10128 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:45731
I20260812 14:44:54.854918 10260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:45731 every 8 connection(s)
I20260812 14:44:54.855199 10128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:54.856074 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:54.858212 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Bootstrap starting.
I20260812 14:44:54.858721 10261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:54.858946 10261 log.cc:824] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:54.859551 10261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: No bootstrap required, opened a new log
I20260812 14:44:54.859642 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 10128
I20260812 14:44:54.859747 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:44:54.859910 16271 external_mini_cluster.cc:1428] Setting key 6c460add954e9c1a39c799e9a9d728df
I20260812 14:44:54.861532 10261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:54.861696 10261 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:54.861770 10261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0b6218bf604053bff2ac2006057e7c, State: Initialized, Role: FOLLOWER
I20260812 14:44:54.861908 10261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:54.861994 10261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:44:54.862046 10261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:44:54.862095 10261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:54.863085 10261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:54.863204 10261 leader_election.cc:302] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e0b6218bf604053bff2ac2006057e7c; no voters:
I20260812 14:44:54.863346 10261 leader_election.cc:288] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:44:54.863385 10270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:54.863516 10270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 LEADER]: Becoming Leader. State: Replica: 7e0b6218bf604053bff2ac2006057e7c, State: Running, Role: LEADER
I20260812 14:44:54.863559 10261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:54.863670 10270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:54.864058 10271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e0b6218bf604053bff2ac2006057e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } } }
I20260812 14:44:54.864073 10273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e0b6218bf604053bff2ac2006057e7c. Latest consensus state: current_term: 1 leader_uuid: "7e0b6218bf604053bff2ac2006057e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } } }
I20260812 14:44:54.864147 10271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:54.864171 10273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:54.864360 10275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:54.864869 10275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:44:54.866590 10275 catalog_manager.cc:1347] Generated new cluster ID: f22fa0b023d149fd83e429027ae4bc7f
I20260812 14:44:54.866638 10275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:44:54 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:54.870440 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:54.860341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60566 (local address 127.15.227.254:45731)
0812 14:44:54.860523 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:54.860531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:54.860567 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:44:54.861500 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:54.861509 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:54.861571 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:54.861727 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:54.862406 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:54.863715 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:54.864736 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:54.865346 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:54.865449 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:54.867664 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:54.867668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:54.867668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:54.867679 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:54.869089 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:54.869393 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:54.869396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:54.869396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:54.869498 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:54.869670 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:54.869671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:54.869671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:54.869878 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:54.870050 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:54.870134 (+ 84us) server_negotiation.cc:299] Negotiation successful
0812 14:44:54.870273 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260812 14:44:54.890854 10275 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:44:54.891327 10275 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:44:54.900339 10275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Generated new TSK 0
I20260812 14:44:54.900470 10275 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:44:54.925765 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:45731
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:55.003930 10317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:55.004117 10317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:55.004146 10317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:55.005236 10317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:55.005280 10317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:55.005314 10317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:44:55.006749 10317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:55.006979 10317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:55.007174 10317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:55.011790 10328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.011778 10334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.011881 10317 server_base.cc:1034] running on GCE node
W20260812 14:44:55.011786 10327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.012212 10317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:55.012408 10317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:55.013540 10317 hybrid_clock.cc:648] HybridClock initialized: now 1786545895013527 us; error 28 us; skew 500 ppm
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:55.015856 10317 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:44:55.016258 10317 webserver.cc:460] Webserver started at http://127.15.227.193:37531/ using document root <none> and password file <none>
I20260812 14:44:55.016459 10317 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:55.016506 10317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:55.016629 10317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:55.017577 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "7ae3bbaa86d94e7daf13978dbe5fced7"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "3ac143652cb89a0fe812bcd75dcdfcdb"
server_key_iv: "82a793b46ac0a28fb92f0a3647cc81b0"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.017892 10317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "7ae3bbaa86d94e7daf13978dbe5fced7"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "3ac143652cb89a0fe812bcd75dcdfcdb"
server_key_iv: "82a793b46ac0a28fb92f0a3647cc81b0"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.019125 10317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:44:55.019758 10347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.019886 10317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:44:55.019966 10317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "7ae3bbaa86d94e7daf13978dbe5fced7"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "3ac143652cb89a0fe812bcd75dcdfcdb"
server_key_iv: "82a793b46ac0a28fb92f0a3647cc81b0"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.020030 10317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:55.040468 10317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:55.042188 10317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:55.042320 10317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:55.042518 10317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:55.042820 10317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:55.042855 10317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.042881 10317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:55.042906 10317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.048380 10317 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:42391
I20260812 14:44:55.048442 10583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:42391 every 8 connection(s)
I20260812 14:44:55.048712 10317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:44:55.050151 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 10317
I20260812 14:44:55.050262 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:44:55.050426 16271 external_mini_cluster.cc:1428] Setting key 10eb694f0692b025c23896fd77e7d6f1
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:55.057588 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.049325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:50103 (local address 127.15.227.254:45731)
0812 14:44:55.049405 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.049407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.049579 (+ 172us) server_negotiation.cc:407] Connection header received
0812 14:44:55.050066 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.050068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.050084 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.050136 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:55.051233 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.051728 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.052774 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.052985 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.053036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.055098 (+ 2062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:55.055099 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:55.055099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:55.055106 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:55.056345 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.056704 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.056705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.056706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.056759 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.056982 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.056982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.056983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.057087 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:55.057143 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.057395 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.057496 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:44:55.057701 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.049289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:50103)
0812 14:44:55.049450 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.049495 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.049954 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.050211 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.050215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.050880 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:55.051132 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.051139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.051807 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.051808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.052669 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.052673 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.053057 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.053058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.053205 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.053637 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:55.053646 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:55.055010 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:55.056416 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.056423 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.056425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.056632 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.056816 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.056817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.056817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.056926 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.057141 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:55.057148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:55.057280 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.057364 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.057505 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260812 14:44:55.058110 10585 heartbeater.cc:347] Connected to a master server at 127.15.227.254:45731
I20260812 14:44:55.058197 10585 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:55.058354 10585 heartbeater.cc:511] Master 127.15.227.254:45731 requested a full tablet report, sending...
I20260812 14:44:55.058809 10194 ts_manager.cc:194] Registered new tserver with Master: 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193:42391)
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
I20260812 14:44:55.059402 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:50103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:44:55.073475 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:45731
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:55.154980 10598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:55.155164 10598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:55.155201 10598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:55.156287 10598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:55.156327 10598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:55.156358 10598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:44:55.157785 10598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:55.158041 10598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:55.158244 10598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:55.162943 10607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.163021 10609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.163106 10598 server_base.cc:1034] running on GCE node
W20260812 14:44:55.163152 10606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.163426 10598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:55.163625 10598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:55.164758 10598 hybrid_clock.cc:648] HybridClock initialized: now 1786545895164746 us; error 29 us; skew 500 ppm
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:55.167037 10598 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:44:55.167405 10598 webserver.cc:460] Webserver started at http://127.15.227.194:45943/ using document root <none> and password file <none>
I20260812 14:44:55.167588 10598 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:55.167634 10598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:55.167755 10598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:55.168658 10598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "04739a9685804b4294603928472c11c5"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "b21a6a272972bb3831677ac98a6382f9"
server_key_iv: "261383a94ca33bdb5f8967b74c6f225b"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.168985 10598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "04739a9685804b4294603928472c11c5"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "b21a6a272972bb3831677ac98a6382f9"
server_key_iv: "261383a94ca33bdb5f8967b74c6f225b"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.170178 10598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:44:55.170794 10621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.170940 10598 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:55.171020 10598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "04739a9685804b4294603928472c11c5"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "b21a6a272972bb3831677ac98a6382f9"
server_key_iv: "261383a94ca33bdb5f8967b74c6f225b"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.171082 10598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:55.185036 10598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:55.186710 10598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:55.186841 10598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:55.187038 10598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:55.187297 10598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:55.187330 10598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.187354 10598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:55.187371 10598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.192577 10598 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:40359
I20260812 14:44:55.192615 10857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:40359 every 8 connection(s)
I20260812 14:44:55.192885 10598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:44:55.197409 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 10598
I20260812 14:44:55.197517 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:44:55.197667 16271 external_mini_cluster.cc:1428] Setting key 9830400d035891121b4d50e3a049a8d3
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:55.201068 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.193527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:56133 (local address 127.15.227.254:45731)
0812 14:44:55.193597 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.193599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.193722 (+ 123us) server_negotiation.cc:407] Connection header received
0812 14:44:55.194140 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.194141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.194151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.194185 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:55.195001 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.195482 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.196363 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.196535 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.196576 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.198551 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:55.198552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:55.198552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:55.198560 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:55.199829 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.200258 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.200259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.200260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.200310 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.200537 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.200538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.200538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.200627 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:55.200671 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.200942 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.201007 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:44:55.201282 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.193489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:56133)
0812 14:44:55.193613 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.193646 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.194062 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.194243 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.194246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.194748 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:55.194934 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.194938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.195540 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.195541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.196305 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.196307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.196583 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.196584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.196671 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.197028 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:55.197035 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:55.198470 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:55.199898 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.199905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.199907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.200178 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.200362 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.200363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.200363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.200486 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.200675 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:55.200679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:55.200830 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.200932 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.201076 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
I20260812 14:44:55.201721 10859 heartbeater.cc:347] Connected to a master server at 127.15.227.254:45731
I20260812 14:44:55.201812 10859 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:55.201963 10859 heartbeater.cc:511] Master 127.15.227.254:45731 requested a full tablet report, sending...
I20260812 14:44:55.202332 10194 ts_manager.cc:194] Registered new tserver with Master: 04739a9685804b4294603928472c11c5 (127.15.227.194:40359)
I20260812 14:44:55.202678 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:56133
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:44:55.220544 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:45731
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:55.298848 10866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:55.299026 10866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:55.299047 10866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:55.300151 10866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:55.300192 10866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:55.300217 10866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:44:55.301605 10866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:55.301782 10866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:55.301946 10866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:55.306576 10878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.306583 10873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.306579 10881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.306798 10866 server_base.cc:1034] running on GCE node
I20260812 14:44:55.306985 10866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:55.307192 10866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:55.308336 10866 hybrid_clock.cc:648] HybridClock initialized: now 1786545895308320 us; error 28 us; skew 500 ppm
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:55.310623 10866 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:44:55.311012 10866 webserver.cc:460] Webserver started at http://127.15.227.195:32969/ using document root <none> and password file <none>
I20260812 14:44:55.311187 10866 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:55.311225 10866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:55.311311 10866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:44:55.312214 10866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "6618b01bde624897af3f7130eeeca82f"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "e3fa29ea6760abf225386bbc1c929297"
server_key_iv: "47980e5b27df17da75c21f5d3821950b"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.312486 10866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "6618b01bde624897af3f7130eeeca82f"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "e3fa29ea6760abf225386bbc1c929297"
server_key_iv: "47980e5b27df17da75c21f5d3821950b"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.313625 10866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:44:55.314282 10894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.314421 10866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:44:55.314502 10866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "6618b01bde624897af3f7130eeeca82f"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "e3fa29ea6760abf225386bbc1c929297"
server_key_iv: "47980e5b27df17da75c21f5d3821950b"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.314574 10866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:55.330615 10866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:55.332130 10866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:55.332247 10866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:55.332422 10866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:55.332695 10866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:55.332736 10866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.332762 10866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:55.332779 10866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.337877 10866 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35255
I20260812 14:44:55.337930 11131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35255 every 8 connection(s)
I20260812 14:44:55.338199 10866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:55.344734 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 10866
I20260812 14:44:55.344854 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:44:55.345016 16271 external_mini_cluster.cc:1428] Setting key c9d003c04d4a81d80f12419636b8b8bd
I20260812 14:44:55.346004 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.338807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:47333 (local address 127.15.227.254:45731)
0812 14:44:55.338862 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.338864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.339016 (+ 152us) server_negotiation.cc:407] Connection header received
0812 14:44:55.339418 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.339419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.339428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.339474 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:55.340278 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.340746 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.341607 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.341776 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.341816 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.343635 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:55.343636 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:55.343637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:55.343643 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:55.344718 (+ 1075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.345096 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.345098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.345098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.345158 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.345403 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.345403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.345404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.345528 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:55.345591 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.345841 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.345918 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:44:55.346205 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.338769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:47333)
0812 14:44:55.338905 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.338942 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.339348 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.339526 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.339529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.340039 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:55.340208 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.340212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.340805 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.340806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.341538 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.341540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.341824 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.341825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.341911 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.342296 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:55.342303 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:55.343548 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:55.344786 (+ 1238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.344792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.344794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.345026 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.345223 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.345224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.345225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.345348 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.345589 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:55.345596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:55.345717 (+ 121us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.345808 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.345946 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260812 14:44:55.346594 11132 heartbeater.cc:347] Connected to a master server at 127.15.227.254:45731
I20260812 14:44:55.346670 11132 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:55.346828 11132 heartbeater.cc:511] Master 127.15.227.254:45731 requested a full tablet report, sending...
I20260812 14:44:55.347121 10194 ts_manager.cc:194] Registered new tserver with Master: 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255)
I20260812 14:44:55.347467 10194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:47333
I20260812 14:44:55.356516 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:44:55.362910 10268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.357759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.15.227.254:45731)
0812 14:44:55.357812 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.357813 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.357905 (+ 92us) server_negotiation.cc:407] Connection header received
0812 14:44:55.357962 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.357963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.357973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.358027 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:55.358529 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.358975 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.359698 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.359880 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.359931 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.360940 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:55.360941 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:55.360941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:55.360948 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:55.361865 (+ 917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.362224 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.362225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.362225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.362274 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.362438 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.362438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.362439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.362571 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:55.362658 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.362761 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.362832 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:44:55.364300 10194 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:44:55.365926 10194 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:44:55.371717 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.367801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:40359 (local address 127.0.0.1:35034)
0812 14:44:55.368001 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.368039 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.368102 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.368585 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.368586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.368588 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:55.368621 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.368622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.369860 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.369861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.370817 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.370820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.371440 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.371441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.371517 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.371529 (+ 12us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.371590 (+ 61us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.371650 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":47,"threads_started":1}
I20260812 14:44:55.371730 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.367891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.15.227.194:40359)
0812 14:44:55.368378 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.368382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.368390 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:44:55.368420 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.368422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.368434 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.368500 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:55.368695 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.369750 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.370889 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.371368 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.371432 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.371486 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.371612 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.371654 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":83,"threads_started":1}
I20260812 14:44:55.372005 11152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.367801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35255 (local address 127.0.0.1:44402)
0812 14:44:55.368023 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.368052 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.368133 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.368345 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.368348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.368351 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:55.368407 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.368409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.369788 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.369789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.370854 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.370856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.371763 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.371763 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.371841 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.371843 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.371888 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.371930 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":4,"thread_start_us":43,"threads_started":1}
I20260812 14:44:55.372030 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.367973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.15.227.195:35255)
0812 14:44:55.368117 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.368122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.368132 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:44:55.368190 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.368192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.368206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.368268 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:55.368622 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.369689 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.370933 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.371606 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.371674 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.371724 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.371920 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.371960 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260812 14:44:55.372088 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.368138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.15.227.193:42391)
0812 14:44:55.368333 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.368337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.368346 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:44:55.368381 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.368383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.368398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.368474 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:55.368609 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.369437 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.370886 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.371606 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.371675 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.371724 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.371973 (+ 249us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.372014 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":117,"threads_started":1}
I20260812 14:44:55.372239 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.367809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42391 (local address 127.0.0.1:59062)
0812 14:44:55.368191 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.368200 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.368235 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.368484 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.368485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.368487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:55.368543 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.368545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.369538 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.369539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.370791 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.370794 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.371761 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.371762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.371833 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.371838 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.371886 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.371930 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":6,"thread_start_us":76,"threads_started":1}
I20260812 14:44:55.372674 10733 tablet_service.cc:1467] Processing CreateTablet for tablet fefd70412dda4f68b01c5d3ccbf2833c (DEFAULT_TABLE table=test-table [id=0630190e80184029a5c6692fa456d9a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:55.372758 10984 tablet_service.cc:1467] Processing CreateTablet for tablet fefd70412dda4f68b01c5d3ccbf2833c (DEFAULT_TABLE table=test-table [id=0630190e80184029a5c6692fa456d9a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:55.372910 10440 tablet_service.cc:1467] Processing CreateTablet for tablet fefd70412dda4f68b01c5d3ccbf2833c (DEFAULT_TABLE table=test-table [id=0630190e80184029a5c6692fa456d9a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:55.373015 10984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fefd70412dda4f68b01c5d3ccbf2833c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:55.373036 10733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fefd70412dda4f68b01c5d3ccbf2833c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:55.373200 10440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fefd70412dda4f68b01c5d3ccbf2833c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:55.375476 11158 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Bootstrap starting.
I20260812 14:44:55.376030 11158 tablet_bootstrap.cc:654] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:55.376300 11160 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Bootstrap starting.
I20260812 14:44:55.376739 11161 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap starting.
I20260812 14:44:55.376749 11158 log.cc:824] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:55.376971 11160 tablet_bootstrap.cc:654] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:55.377437 11161 tablet_bootstrap.cc:654] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:55.377472 11160 log.cc:824] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:55.377532 11158 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: No bootstrap required, opened a new log
I20260812 14:44:55.377610 11158 ts_tablet_manager.cc:1397] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:55.377986 11161 log.cc:824] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:55.378147 11160 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: No bootstrap required, opened a new log
I20260812 14:44:55.378207 11160 ts_tablet_manager.cc:1397] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:44:55.379086 11158 raft_consensus.cc:348] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.379208 11158 raft_consensus.cc:374] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:55.379228 11158 raft_consensus.cc:729] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6618b01bde624897af3f7130eeeca82f, State: Initialized, Role: FOLLOWER
I20260812 14:44:55.379328 11158 consensus_queue.cc:260] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.379570 11158 ts_tablet_manager.cc:1428] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:44:55.379611 11132 heartbeater.cc:503] Master 127.15.227.254:45731 was elected leader, sending a full tablet report...
I20260812 14:44:55.379612 11160 raft_consensus.cc:348] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.379755 11160 raft_consensus.cc:374] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:55.379786 11160 raft_consensus.cc:729] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04739a9685804b4294603928472c11c5, State: Initialized, Role: FOLLOWER
I20260812 14:44:55.379904 11160 consensus_queue.cc:260] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.380158 11160 ts_tablet_manager.cc:1428] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:44:55.380229 10859 heartbeater.cc:503] Master 127.15.227.254:45731 was elected leader, sending a full tablet report...
I20260812 14:44:55.380749 11161 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: No bootstrap required, opened a new log
I20260812 14:44:55.380826 11161 ts_tablet_manager.cc:1397] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:44:55.382275 11161 raft_consensus.cc:348] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.382393 11161 raft_consensus.cc:374] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:55.382413 11161 raft_consensus.cc:729] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae3bbaa86d94e7daf13978dbe5fced7, State: Initialized, Role: FOLLOWER
I20260812 14:44:55.382514 11161 consensus_queue.cc:260] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.382747 11161 ts_tablet_manager.cc:1428] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:44:55.382807 10585 heartbeater.cc:503] Master 127.15.227.254:45731 was elected leader, sending a full tablet report...
W20260812 14:44:55.443404 10861 tablet.cc:2369] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:44:55.515980 11173 raft_consensus.cc:482] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:44:55.516099 11173 raft_consensus.cc:504] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.516454 11173 leader_election.cc:288] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255), 04739a9685804b4294603928472c11c5 (127.15.227.194:40359)
I20260812 14:44:55.519310 11153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.516669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:38771 (local address 127.15.227.194:40359)
0812 14:44:55.516716 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.516718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.516884 (+ 166us) server_negotiation.cc:407] Connection header received
0812 14:44:55.516965 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.516966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.516976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.517029 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:55.517169 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.517679 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.518649 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.519020 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.519084 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.519091 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.519204 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.519240 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:44:55.519323 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.516795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:44749 (local address 127.15.227.195:35255)
0812 14:44:55.516856 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.516868 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.516877 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:44:55.516903 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.516904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.516912 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.516962 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:55.517110 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.517605 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.518626 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.519004 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.519059 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.519067 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.519219 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.519262 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 14:44:55.519333 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.516595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35255 (local address 127.15.227.193:44749)
0812 14:44:55.516671 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.516681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.516718 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.516969 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.516970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.516972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:55.517048 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.517050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.517666 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.517667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.518549 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.518551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.519073 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.519074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.519155 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.519158 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.519209 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.519246 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 14:44:55.519343 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.516618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:40359 (local address 127.15.227.193:38771)
0812 14:44:55.516812 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.516821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.516893 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.517038 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.517039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.517040 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:55.517107 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.517108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.517759 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.517760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.518564 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.518566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.519093 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.519094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.519145 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.519149 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.519190 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.519228 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":117,"threads_started":1}
I20260812 14:44:55.519685 11030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fefd70412dda4f68b01c5d3ccbf2833c" candidate_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6618b01bde624897af3f7130eeeca82f" is_pre_election: true
I20260812 14:44:55.519701 10764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fefd70412dda4f68b01c5d3ccbf2833c" candidate_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04739a9685804b4294603928472c11c5" is_pre_election: true
I20260812 14:44:55.519830 10764 raft_consensus.cc:2454] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ae3bbaa86d94e7daf13978dbe5fced7 in term 0.
I20260812 14:44:55.519830 11030 raft_consensus.cc:2454] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ae3bbaa86d94e7daf13978dbe5fced7 in term 0.
I20260812 14:44:55.520047 10354 leader_election.cc:302] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04739a9685804b4294603928472c11c5, 7ae3bbaa86d94e7daf13978dbe5fced7; no voters:
I20260812 14:44:55.520185 11173 raft_consensus.cc:2781] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:44:55.520231 11173 raft_consensus.cc:482] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:44:55.520257 11173 raft_consensus.cc:3037] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:55.520947 11173 raft_consensus.cc:504] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.521059 11173 leader_election.cc:288] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 1 election: Requested vote from peers 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255), 04739a9685804b4294603928472c11c5 (127.15.227.194:40359)
I20260812 14:44:55.521194 11030 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fefd70412dda4f68b01c5d3ccbf2833c" candidate_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6618b01bde624897af3f7130eeeca82f"
I20260812 14:44:55.521209 10764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fefd70412dda4f68b01c5d3ccbf2833c" candidate_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04739a9685804b4294603928472c11c5"
I20260812 14:44:55.521278 11030 raft_consensus.cc:3037] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:55.521288 10764 raft_consensus.cc:3037] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:55.521975 11030 raft_consensus.cc:2454] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ae3bbaa86d94e7daf13978dbe5fced7 in term 1.
I20260812 14:44:55.522087 10764 raft_consensus.cc:2454] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ae3bbaa86d94e7daf13978dbe5fced7 in term 1.
I20260812 14:44:55.522146 10350 leader_election.cc:302] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6618b01bde624897af3f7130eeeca82f, 7ae3bbaa86d94e7daf13978dbe5fced7; no voters:
I20260812 14:44:55.522275 11173 raft_consensus.cc:2781] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:55.522385 11173 raft_consensus.cc:686] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 LEADER]: Becoming Leader. State: Replica: 7ae3bbaa86d94e7daf13978dbe5fced7, State: Running, Role: LEADER
I20260812 14:44:55.522470 11173 consensus_queue.cc:237] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.523203 10194 catalog_manager.cc:5495] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193). New cstate: current_term: 1 leader_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } health_report { overall_health: UNKNOWN } } }
W20260812 14:44:55.549353 10588 tablet.cc:2369] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:44:55.550869 11157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.548891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.15.227.193:42391)
0812 14:44:55.548933 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.548935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.548942 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:44:55.548966 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.548967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.548982 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.549034 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:44:55.549147 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.549588 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.550317 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.550489 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.550535 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.550636 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:44:55.550721 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:44:55.550781 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.550792 (+ 11us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.550806 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:44:55.552332 10764 raft_consensus.cc:1261] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 1 FOLLOWER]: Refusing update from remote peer 7ae3bbaa86d94e7daf13978dbe5fced7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:55.552332 11030 raft_consensus.cc:1261] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Refusing update from remote peer 7ae3bbaa86d94e7daf13978dbe5fced7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:55.552696 11182 consensus_queue.cc:1035] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:55.552814 11173 consensus_queue.cc:1035] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:55.554768 11188 mvcc.cc:205] Tried to move back new op lower bound from 7317691988180164608 to 7317691988060504064. Current Snapshot: MvccSnapshot[applied={T|T < 7317691988180164608}]
I20260812 14:44:55.555059 11191 mvcc.cc:205] Tried to move back new op lower bound from 7317691988180164608 to 7317691988060504064. Current Snapshot: MvccSnapshot[applied={T|T < 7317691988180164608}]
I20260812 14:44:55.555718 11189 mvcc.cc:205] Tried to move back new op lower bound from 7317691988180164608 to 7317691988060504064. Current Snapshot: MvccSnapshot[applied={T|T < 7317691988180164608}]
I20260812 14:44:55.556865 10194 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:55.556975 10194 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:55.557647 10194 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Sending DeleteTablet for 3 replicas of tablet fefd70412dda4f68b01c5d3ccbf2833c
I20260812 14:44:55.557943 10440 tablet_service.cc:1514] Processing DeleteTablet for tablet fefd70412dda4f68b01c5d3ccbf2833c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:55 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:59062
I20260812 14:44:55.558009 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 10317
I20260812 14:44:55.558478 10733 tablet_service.cc:1514] Processing DeleteTablet for tablet fefd70412dda4f68b01c5d3ccbf2833c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:55 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:35034
I20260812 14:44:55.560990 11208 tablet_replica.cc:330] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: stopping tablet replica
I20260812 14:44:55.561069 10984 tablet_service.cc:1514] Processing DeleteTablet for tablet fefd70412dda4f68b01c5d3ccbf2833c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:55 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:44402
I20260812 14:44:55.561088 11208 raft_consensus.cc:2229] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:44:55.561188 11208 raft_consensus.cc:2258] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:55.563274 11210 tablet_replica.cc:330] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: stopping tablet replica
I20260812 14:44:55.563359 11210 raft_consensus.cc:2229] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:44:55.563436 11210 raft_consensus.cc:2258] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:55.565788 11210 ts_tablet_manager.cc:1905] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:55.567145 11210 ts_tablet_manager.cc:1918] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:55.567204 11210 log.cc:1196] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/fefd70412dda4f68b01c5d3ccbf2833c
I20260812 14:44:55.567431 11210 ts_tablet_manager.cc:1939] T fefd70412dda4f68b01c5d3ccbf2833c P 6618b01bde624897af3f7130eeeca82f: Deleting consensus metadata
I20260812 14:44:55.568185 11208 ts_tablet_manager.cc:1905] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:55.568521 10169 catalog_manager.cc:4841] TS 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255): tablet fefd70412dda4f68b01c5d3ccbf2833c (table test-table [id=0630190e80184029a5c6692fa456d9a2]) successfully deleted
I20260812 14:44:55.569657 11208 ts_tablet_manager.cc:1918] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:55.569729 11208 log.cc:1196] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/fefd70412dda4f68b01c5d3ccbf2833c
I20260812 14:44:55.569962 11208 ts_tablet_manager.cc:1939] T fefd70412dda4f68b01c5d3ccbf2833c P 04739a9685804b4294603928472c11c5: Deleting consensus metadata
I20260812 14:44:55.570341 10174 catalog_manager.cc:4841] TS 04739a9685804b4294603928472c11c5 (127.15.227.194:40359): tablet fefd70412dda4f68b01c5d3ccbf2833c (table test-table [id=0630190e80184029a5c6692fa456d9a2]) successfully deleted
W20260812 14:44:55.573249 10167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:42391 (error 108)
I20260812 14:44:55.573473 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 10598
I20260812 14:44:55.577476 11150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.573554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42391 (local address 127.0.0.1:59088)
0812 14:44:55.577405 (+ 3851us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.577448 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3825}
W20260812 14:44:55.579228 10167 catalog_manager.cc:4584] TS 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193:42391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fefd70412dda4f68b01c5d3ccbf2833c: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42391: connect: Connection refused (error 111)
I20260812 14:44:55.585542 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 10866
I20260812 14:44:55.597978 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 10128
I20260812 14:44:55.608887 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45731
--webserver_interface=127.15.227.254
--webserver_port=37915
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:45731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:55.693684 11214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:55.693881 11214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:55.693914 11214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:55.695062 11214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:55.695109 11214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:55.695132 11214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:55.695150 11214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:55.695170 11214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:55.696729 11214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:45731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=37915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:55.696974 11214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:55.697214 11214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:55.706506 11225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.706514 11226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.706517 11229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.706880 11214 server_base.cc:1034] running on GCE node
I20260812 14:44:55.707130 11214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:55.707408 11214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:55.708544 11214 hybrid_clock.cc:648] HybridClock initialized: now 1786545895708531 us; error 28 us; skew 500 ppm
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:55.710881 11214 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:55.711295 11214 webserver.cc:460] Webserver started at http://127.15.227.254:37915/ using document root <none> and password file <none>
I20260812 14:44:55.711494 11214 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:55.711555 11214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:55.712777 11214 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:55.713357 11239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.713521 11214 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:55.713609 11214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "7e0b6218bf604053bff2ac2006057e7c"
format_stamp: "Formatted at 2026-08-12 14:44:54 on dist-test-slave-q9h9"
server_key: "466c20f7bf64b63013edb3c383fd02f5"
server_key_iv: "8252f38dbc75371df4d1715f08dbff98"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.713966 11214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:55.731067 11214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:55.732971 11214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:55.733095 11214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:55.736984 11214 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:45731
I20260812 14:44:55.737013 11356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:45731 every 8 connection(s)
I20260812 14:44:55.737347 11214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:55.738871 11358 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:44:55.739454 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Bootstrap starting.
I20260812 14:44:55.746450 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 11214
I20260812 14:44:55.746791 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:42391
--local_ip_for_outbound_sockets=127.15.227.193
--tserver_master_addrs=127.15.227.254:45731
--webserver_port=37531
--webserver_interface=127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:55.748329 11358 log.cc:824] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:55.750131 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:44:55.750375 11358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Bootstrap complete.
I20260812 14:44:55.752463 11358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:55.752668 11358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0b6218bf604053bff2ac2006057e7c, State: Initialized, Role: FOLLOWER
I20260812 14:44:55.752817 11358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:55.752877 11358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:44:55.752923 11358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:44:55.752969 11358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:44:55.753959 11358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:55.754068 11358 leader_election.cc:302] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e0b6218bf604053bff2ac2006057e7c; no voters:
I20260812 14:44:55.754212 11358 leader_election.cc:288] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:44:55.754262 11363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:44:55.754372 11363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 LEADER]: Becoming Leader. State: Replica: 7e0b6218bf604053bff2ac2006057e7c, State: Running, Role: LEADER
I20260812 14:44:55.754460 11358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:55.754468 11363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:55.755191 11364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7e0b6218bf604053bff2ac2006057e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } } }
I20260812 14:44:55.755199 11366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e0b6218bf604053bff2ac2006057e7c. Latest consensus state: current_term: 2 leader_uuid: "7e0b6218bf604053bff2ac2006057e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } } }
I20260812 14:44:55.755287 11364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:55.755291 11366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:55.755494 11383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:55.756183 11383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:44:55.756295 11383 catalog_manager.cc:1259] Loaded cluster ID: f22fa0b023d149fd83e429027ae4bc7f
I20260812 14:44:55.756332 11383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:44:55.756554 11383 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 14:44:55.760442 11383 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 14:44:55.762717 11383 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:44:55.762903 11383 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Loaded TSK: 0
I20260812 14:44:55.763103 11383 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:44:55.829993 11360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:55.830204 11360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:55.830226 11360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:55.831324 11360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:55.831362 11360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:55.831385 11360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:44:55.832758 11360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:42391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=37531
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:55.832959 11360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:55.833169 11360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:55.837661 11404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.837777 11412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.837675 11416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.837972 11360 server_base.cc:1034] running on GCE node
I20260812 14:44:55.838184 11360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:55.838388 11360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:55.839547 11360 hybrid_clock.cc:648] HybridClock initialized: now 1786545895839530 us; error 34 us; skew 500 ppm
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:55.841955 11360 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:44:55.842420 11360 webserver.cc:460] Webserver started at http://127.15.227.193:37531/ using document root <none> and password file <none>
I20260812 14:44:55.842638 11360 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:55.842700 11360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:55.843922 11360 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:55.844553 11427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.844724 11360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.844803 11360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "7ae3bbaa86d94e7daf13978dbe5fced7"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "3ac143652cb89a0fe812bcd75dcdfcdb"
server_key_iv: "82a793b46ac0a28fb92f0a3647cc81b0"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.845099 11360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:55.861749 11360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:55.863713 11360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:55.863824 11360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:55.864018 11360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:55.864421 11437 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:44:55.866289 11360 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:44:55.866330 11360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:44:55.866360 11360 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:44:55.866930 11360 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:44:55.866959 11360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:44:55.867061 11437 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap starting.
I20260812 14:44:55.872427 11360 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:42391
I20260812 14:44:55.872761 11360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:44:55.874220 11660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:42391 every 8 connection(s)
I20260812 14:44:55.877810 11437 log.cc:824] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:55.879997 11437 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:44:55.880263 11437 tablet_bootstrap.cc:492] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap complete.
I20260812 14:44:55.881072 11437 ts_tablet_manager.cc:1397] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260812 14:44:55.881987 11437 raft_consensus.cc:348] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.882259 11437 raft_consensus.cc:729] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae3bbaa86d94e7daf13978dbe5fced7, State: Initialized, Role: FOLLOWER
I20260812 14:44:55.882393 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 11360
I20260812 14:44:55.882412 11437 consensus_queue.cc:260] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } } peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } }
I20260812 14:44:55.882668 11437 ts_tablet_manager.cc:1428] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:44:55.882671 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:40359
--local_ip_for_outbound_sockets=127.15.227.194
--tserver_master_addrs=127.15.227.254:45731
--webserver_port=45943
--webserver_interface=127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:55.887459 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.873437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46885 (local address 127.15.227.254:45731)
0812 14:44:55.878127 (+ 4690us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.878136 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.878171 (+ 35us) server_negotiation.cc:407] Connection header received
0812 14:44:55.878306 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.878313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.878373 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.878555 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:55.879161 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.879993 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.880998 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.881425 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.881520 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.884072 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:55.884077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:55.884077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:55.884088 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:55.885786 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.886256 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.886258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.886259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.886345 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.886586 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.886586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.886586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.886785 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:55.886840 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.887178 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.887289 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4639,"thread_start_us":81,"threads_started":1}
I20260812 14:44:55.887493 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.873375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:46885)
0812 14:44:55.873539 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.873586 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.873974 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.878601 (+ 4627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.878605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.878866 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:55.879043 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.879048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.880106 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.880108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.880918 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.880921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.881526 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.881528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.881660 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.882158 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:55.882166 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:55.883948 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:55.885863 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.885872 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.885874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.886185 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.886405 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.886405 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.886406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.886538 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.886840 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:55.886846 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:55.887013 (+ 167us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.887137 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.887302 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260812 14:44:55.888206 11661 heartbeater.cc:347] Connected to a master server at 127.15.227.254:45731
I20260812 14:44:55.888300 11661 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:55.888463 11661 heartbeater.cc:511] Master 127.15.227.254:45731 requested a full tablet report, sending...
I20260812 14:44:55.889145 11270 ts_manager.cc:194] Registered new tserver with Master: 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193:42391)
I20260812 14:44:55.890218 11270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:46885
I20260812 14:44:55.891134 11661 heartbeater.cc:503] Master 127.15.227.254:45731 was elected leader, sending a full tablet report...
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:44:55.897527 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.889730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59092 (local address 127.15.227.193:42391)
0812 14:44:55.890096 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:55.890100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:55.890109 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:44:55.890139 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:55.890140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:55.890162 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:55.890224 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:55.890736 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.891683 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.892722 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.893009 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.893063 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.895049 (+ 1986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:55.895057 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:55.895057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:55.895064 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:55.896392 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.896737 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.896741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.896741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.896803 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:55.896972 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:55.896973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:55.896973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:55.897102 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:55.897173 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:55.897347 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 14:44:55.897435 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":81,"threads_started":1}
I20260812 14:44:55.897531 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:55.889676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42391 (local address 127.0.0.1:59092)
0812 14:44:55.889868 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:55.889896 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:55.889952 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:55.890247 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:55.890255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:55.890578 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:55.890649 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.890652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.891754 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.891755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.892634 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:55.892637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:55.893081 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:55.893082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:55.893163 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:55.893568 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:55.893573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:55.894965 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:55.896472 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.896478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.896480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.896666 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.896861 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:55.896862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:55.896863 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:55.896923 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:55.897166 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:55.897172 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:55.897223 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 14:44:55.897302 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 14:44:55.897441 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":80,"threads_started":1}
I20260812 14:44:55.897881 11526 tablet_service.cc:1514] Processing DeleteTablet for tablet fefd70412dda4f68b01c5d3ccbf2833c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:55 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:59092
I20260812 14:44:55.898387 11682 tablet_replica.cc:330] stopping tablet replica
I20260812 14:44:55.898479 11682 raft_consensus.cc:2229] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:44:55.898542 11682 raft_consensus.cc:2258] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:55.898826 11682 ts_tablet_manager.cc:1905] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:55.900530 11682 ts_tablet_manager.cc:1918] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:55.900588 11682 log.cc:1196] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/fefd70412dda4f68b01c5d3ccbf2833c
I20260812 14:44:55.900797 11682 ts_tablet_manager.cc:1939] T fefd70412dda4f68b01c5d3ccbf2833c P 7ae3bbaa86d94e7daf13978dbe5fced7: Deleting consensus metadata
I20260812 14:44:55.901249 11242 catalog_manager.cc:4841] TS 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193:42391): tablet fefd70412dda4f68b01c5d3ccbf2833c (table test-table [id=0630190e80184029a5c6692fa456d9a2]) successfully deleted
W20260812 14:44:55.966146 11674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:55.966321 11674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:55.966341 11674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:55.967401 11674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:55.967430 11674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:55.967454 11674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:44:55.968902 11674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:40359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=45943
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:55.969074 11674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:55.969247 11674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:55.973891 11693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:55.973870 11689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.973925 11674 server_base.cc:1034] running on GCE node
W20260812 14:44:55.973863 11690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:55.974246 11674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:55.974450 11674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:55.975580 11674 hybrid_clock.cc:648] HybridClock initialized: now 1786545895975569 us; error 28 us; skew 500 ppm
Aug 12 14:44:55 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:55.977883 11674 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:44:55.978319 11674 webserver.cc:460] Webserver started at http://127.15.227.194:45943/ using document root <none> and password file <none>
I20260812 14:44:55.978528 11674 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:55.978583 11674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:55.979800 11674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:44:55.980309 11710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.980441 11674 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:44:55.980512 11674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "04739a9685804b4294603928472c11c5"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "b21a6a272972bb3831677ac98a6382f9"
server_key_iv: "261383a94ca33bdb5f8967b74c6f225b"
server_key_version: "encryptionkey@0"
I20260812 14:44:55.980798 11674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:55.993049 11674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:55.994973 11674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:55.995095 11674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:55.995290 11674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:55.995549 11674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:55.995584 11674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:55.995608 11674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:55.995633 11674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.001032 11674 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:40359
I20260812 14:44:56.001066 11915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:40359 every 8 connection(s)
I20260812 14:44:56.001358 11674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:44:56 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545895, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:56.008890 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 11674
I20260812 14:44:56.009193 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:35255
--local_ip_for_outbound_sockets=127.15.227.195
--tserver_master_addrs=127.15.227.254:45731
--webserver_port=32969
--webserver_interface=127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:56.009835 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.001992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:58201 (local address 127.15.227.254:45731)
0812 14:44:56.002048 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.002050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.002249 (+ 199us) server_negotiation.cc:407] Connection header received
0812 14:44:56.002690 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.002692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.002710 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.002761 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:56.003622 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.004127 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.005080 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.005289 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.005337 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.007200 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:56.007201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:56.007202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:56.007209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:56.008344 (+ 1135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:56.008710 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:56.008711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:56.008711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:56.008779 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:56.009089 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:56.009090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:56.009090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:56.009228 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:56.009288 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.009634 (+ 346us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.009731 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:44:56.009938 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.001975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:58201)
0812 14:44:56.002140 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.002172 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:56.002591 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:56.002835 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:56.002838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:56.003353 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:56.003525 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.003529 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.004227 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.004228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.005009 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.005012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.005352 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.005353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.005452 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.005806 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:56.005814 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:56.007112 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:56.008408 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:56.008414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:56.008415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:56.008640 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:56.008862 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:56.008864 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:56.008864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:56.009002 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:56.009341 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:56.009348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:56.009494 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 14:44:56.009596 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 14:44:56.009742 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260812 14:44:56.010542 11916 heartbeater.cc:347] Connected to a master server at 127.15.227.254:45731
I20260812 14:44:56.010629 11916 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:56.010802 11916 heartbeater.cc:511] Master 127.15.227.254:45731 requested a full tablet report, sending...
I20260812 14:44:56.011152 11270 ts_manager.cc:194] Registered new tserver with Master: 04739a9685804b4294603928472c11c5 (127.15.227.194:40359)
I20260812 14:44:56.011511 11270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:58201
W20260812 14:44:56.089793 11920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:56.089977 11920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:56.089999 11920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:56.091126 11920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:56.091166 11920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:56.091190 11920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:44:56.092547 11920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:35255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=32969
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:56.092711 11920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:56.092876 11920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:44:56.097438 11920 server_base.cc:1034] running on GCE node
W20260812 14:44:56.097417 11931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:56.097429 11933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:56.097424 11939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.097787 11920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:56.098004 11920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:56.099171 11920 hybrid_clock.cc:648] HybridClock initialized: now 1786545896099159 us; error 29 us; skew 500 ppm
Aug 12 14:44:56 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:56.101418 11920 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:44:56.101831 11920 webserver.cc:460] Webserver started at http://127.15.227.195:32969/ using document root <none> and password file <none>
I20260812 14:44:56.102041 11920 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:56.102113 11920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:56.103363 11920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:44:56.103946 11955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.104125 11920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:44:56.104205 11920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "6618b01bde624897af3f7130eeeca82f"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "e3fa29ea6760abf225386bbc1c929297"
server_key_iv: "47980e5b27df17da75c21f5d3821950b"
server_key_version: "encryptionkey@0"
I20260812 14:44:56.104538 11920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:56.122468 11920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:56.124145 11920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:56.124267 11920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:56.124457 11920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:56.124737 11920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:56.124769 11920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.124795 11920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:56.124811 11920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.130424 11920 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35255
I20260812 14:44:56.130458 12172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35255 every 8 connection(s)
I20260812 14:44:56.130769 11920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:44:56.134424 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 11920
Aug 12 14:44:56 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:44:56.139750 11669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.131397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:50651 (local address 127.15.227.254:45731)
0812 14:44:56.131455 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.131457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.131629 (+ 172us) server_negotiation.cc:407] Connection header received
0812 14:44:56.132085 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.132086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.132099 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.132161 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:56.133006 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.133493 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.134462 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.134730 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.134805 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.136990 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:56.136991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:56.136992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:56.136998 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:56.138354 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:56.138762 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:56.138763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:56.138764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:56.138836 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:56.139127 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:56.139129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:56.139129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:56.139239 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:56.139302 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.139543 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.139659 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:44:56.139853 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.131384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:50651)
0812 14:44:56.131521 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.131552 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:56.131988 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:56.132226 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:56.132229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:56.132746 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:56.132916 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.132920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.133569 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.133571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.134387 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.134389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.134873 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.134874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.134996 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.135408 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:56.135416 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:56.136907 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:56.138425 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:56.138432 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:56.138434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:56.138696 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:56.138925 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:56.138926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:56.138926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:56.139034 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:56.139302 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:56.139309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:56.139427 (+ 118us) client_negotiation.cc:769] Sending connection context
0812 14:44:56.139512 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 14:44:56.139658 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260812 14:44:56.140272 12173 heartbeater.cc:347] Connected to a master server at 127.15.227.254:45731
I20260812 14:44:56.140358 12173 heartbeater.cc:464] Registering TS with master...
I20260812 14:44:56.140550 12173 heartbeater.cc:511] Master 127.15.227.254:45731 requested a full tablet report, sending...
I20260812 14:44:56.140959 11270 ts_manager.cc:194] Registered new tserver with Master: 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255)
I20260812 14:44:56.141177 12179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.134809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60592 (local address 127.15.227.254:45731)
0812 14:44:56.134914 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.134916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.134986 (+ 70us) server_negotiation.cc:407] Connection header received
0812 14:44:56.135048 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.135049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.135059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.135132 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:56.135650 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.136264 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.137165 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.137416 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.137480 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.138734 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:56.138736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:56.138736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:56.138743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:56.140102 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:56.140444 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:56.140445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:56.140446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:56.140506 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:56.140687 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:56.140687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:56.140688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:56.140806 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:56.140956 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.141013 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.141103 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":59,"threads_started":1}
I20260812 14:44:56.141458 11270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:50651
I20260812 14:44:56.141799 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:44:56.144048 12179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.142133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.15.227.254:45731)
0812 14:44:56.142171 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.142172 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.142255 (+ 83us) server_negotiation.cc:407] Connection header received
0812 14:44:56.142325 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.142325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.142351 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.142401 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:44:56.142462 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.142876 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.143537 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.143708 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.143749 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.143842 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:44:56.143921 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:44:56.143966 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.143978 (+ 12us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.143993 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:44:56.144310 11268 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:60594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:44:56.145766 11268 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:44:56.147926 11526 tablet_service.cc:1467] Processing CreateTablet for tablet 3a927b75dd404ff38bd6f2041d3cb42c (DEFAULT_TABLE table=test-table [id=6afe84959b5b49e49592c7c27eb7311d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:56.148141 11526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a927b75dd404ff38bd6f2041d3cb42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:56.149718 11437 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap starting.
I20260812 14:44:56.150067 11437 tablet_bootstrap.cc:654] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:56.150640 11437 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: No bootstrap required, opened a new log
I20260812 14:44:56.150694 11437 ts_tablet_manager.cc:1397] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260812 14:44:56.151023 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.147509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:40359 (local address 127.0.0.1:35042)
0812 14:44:56.147558 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.147567 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:56.147602 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:56.147963 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:56.147963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:56.147964 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:56.148006 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.148007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.149032 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.149034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.149859 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.149861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.150767 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.150768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.150834 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.150859 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 14:44:56.150922 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:44:56.150967 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260812 14:44:56.151059 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.147608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35042 (local address 127.15.227.194:40359)
0812 14:44:56.147807 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.147811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.147823 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:44:56.147859 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.147861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.147876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.147961 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:56.148101 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.148947 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.149932 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.150699 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.150771 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.150825 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.150937 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.150979 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":93,"threads_started":1}
I20260812 14:44:56.151026 11437 raft_consensus.cc:348] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.151098 11437 raft_consensus.cc:374] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:56.151145 11437 raft_consensus.cc:729] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae3bbaa86d94e7daf13978dbe5fced7, State: Initialized, Role: FOLLOWER
I20260812 14:44:56.151191 11437 consensus_queue.cc:260] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.151268 11437 ts_tablet_manager.cc:1428] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260812 14:44:56.151714 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.147516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35255 (local address 127.0.0.1:44412)
0812 14:44:56.147798 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.147807 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:56.147847 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:56.148301 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:56.148302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:56.148304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:56.148341 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.148342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.149604 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.149605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.150614 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.150617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.151491 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.151492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.151552 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.151556 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:44:56.151603 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:44:56.151651 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":128,"thread_start_us":82,"threads_started":1}
I20260812 14:44:56.151717 11798 tablet_service.cc:1467] Processing CreateTablet for tablet 3a927b75dd404ff38bd6f2041d3cb42c (DEFAULT_TABLE table=test-table [id=6afe84959b5b49e49592c7c27eb7311d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:56.151927 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.147609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44412 (local address 127.15.227.195:35255)
0812 14:44:56.148113 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.148117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.148127 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:44:56.148159 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.148161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.148175 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.148245 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:56.148492 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.149494 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.150699 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.151345 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.151409 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.151456 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.151820 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.151859 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":91,"threads_started":1}
I20260812 14:44:56.152050 11798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a927b75dd404ff38bd6f2041d3cb42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:56.152541 12044 tablet_service.cc:1467] Processing CreateTablet for tablet 3a927b75dd404ff38bd6f2041d3cb42c (DEFAULT_TABLE table=test-table [id=6afe84959b5b49e49592c7c27eb7311d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:44:56.152870 12044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a927b75dd404ff38bd6f2041d3cb42c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:44:56.155498 12191 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Bootstrap starting.
I20260812 14:44:56.156116 12191 tablet_bootstrap.cc:654] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:56.156229 12194 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Bootstrap starting.
I20260812 14:44:56.156822 12191 log.cc:824] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:56.156878 12194 tablet_bootstrap.cc:654] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Neither blocks nor log segments found. Creating new log.
I20260812 14:44:56.157339 12194 log.cc:824] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:56.157610 12191 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: No bootstrap required, opened a new log
I20260812 14:44:56.157670 12191 ts_tablet_manager.cc:1397] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:44:56.157943 12194 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: No bootstrap required, opened a new log
I20260812 14:44:56.157994 12194 ts_tablet_manager.cc:1397] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:44:56.159190 12191 raft_consensus.cc:348] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.159327 12191 raft_consensus.cc:374] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:56.159358 12191 raft_consensus.cc:729] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04739a9685804b4294603928472c11c5, State: Initialized, Role: FOLLOWER
I20260812 14:44:56.159466 12191 consensus_queue.cc:260] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.159427 12194 raft_consensus.cc:348] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.159543 12194 raft_consensus.cc:374] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:44:56.159564 12194 raft_consensus.cc:729] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6618b01bde624897af3f7130eeeca82f, State: Initialized, Role: FOLLOWER
I20260812 14:44:56.159689 12194 consensus_queue.cc:260] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.159727 12191 ts_tablet_manager.cc:1428] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:44:56.159765 11916 heartbeater.cc:503] Master 127.15.227.254:45731 was elected leader, sending a full tablet report...
I20260812 14:44:56.159956 12194 ts_tablet_manager.cc:1428] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:44:56.159965 12173 heartbeater.cc:503] Master 127.15.227.254:45731 was elected leader, sending a full tablet report...
I20260812 14:44:56.178962 12197 raft_consensus.cc:482] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:44:56.179049 12197 raft_consensus.cc:504] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.179339 12197 leader_election.cc:288] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255), 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193:42391)
I20260812 14:44:56.182652 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.179460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35255 (local address 127.15.227.194:39221)
0812 14:44:56.179524 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.179533 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:56.179570 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:56.179684 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:56.179685 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:56.179686 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:56.179744 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.179745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.180562 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.180562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.181670 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.181673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.182390 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.182391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.182478 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.182482 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:44:56.182530 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:44:56.182586 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":3}
I20260812 14:44:56.182794 12199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.179472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42391 (local address 127.15.227.194:46545)
0812 14:44:56.179607 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.179615 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:56.179671 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:56.179809 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:56.179810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:56.179812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:44:56.179868 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.179868 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.180687 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.180688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.181769 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:56.181771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.182530 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.182531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:56.182591 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.182594 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:44:56.182701 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 14:44:56.182734 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260812 14:44:56.182907 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.179531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:46545 (local address 127.15.227.193:42391)
0812 14:44:56.179570 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.179571 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.179675 (+ 104us) server_negotiation.cc:407] Connection header received
0812 14:44:56.179721 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.179722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.179741 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.179796 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:56.179921 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.180624 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.181837 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.182462 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.182525 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.182601 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.182804 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.182847 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:44:56.182945 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.179515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:39221 (local address 127.15.227.195:35255)
0812 14:44:56.179551 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.179554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.179563 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:44:56.179620 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.179621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.179628 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.179682 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:44:56.179799 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.180495 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.181758 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.182317 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.182389 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.182396 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.182689 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.182805 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:44:56.183149 12085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a927b75dd404ff38bd6f2041d3cb42c" candidate_uuid: "04739a9685804b4294603928472c11c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6618b01bde624897af3f7130eeeca82f" is_pre_election: true
I20260812 14:44:56.183151 11567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a927b75dd404ff38bd6f2041d3cb42c" candidate_uuid: "04739a9685804b4294603928472c11c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" is_pre_election: true
I20260812 14:44:56.183267 11567 raft_consensus.cc:2454] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04739a9685804b4294603928472c11c5 in term 0.
I20260812 14:44:56.183274 12085 raft_consensus.cc:2454] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04739a9685804b4294603928472c11c5 in term 0.
I20260812 14:44:56.183492 11713 leader_election.cc:302] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04739a9685804b4294603928472c11c5, 6618b01bde624897af3f7130eeeca82f; no voters:
I20260812 14:44:56.183655 12197 raft_consensus.cc:2781] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:44:56.183712 12197 raft_consensus.cc:482] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:44:56.183733 12197 raft_consensus.cc:3037] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:56.184340 12197 raft_consensus.cc:504] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.184522 12197 leader_election.cc:288] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [CANDIDATE]: Term 1 election: Requested vote from peers 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255), 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193:42391)
I20260812 14:44:56.184613 12085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a927b75dd404ff38bd6f2041d3cb42c" candidate_uuid: "04739a9685804b4294603928472c11c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6618b01bde624897af3f7130eeeca82f"
I20260812 14:44:56.184635 11567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a927b75dd404ff38bd6f2041d3cb42c" candidate_uuid: "04739a9685804b4294603928472c11c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7"
I20260812 14:44:56.184670 12085 raft_consensus.cc:3037] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:56.184695 11567 raft_consensus.cc:3037] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:44:56.185253 12085 raft_consensus.cc:2454] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04739a9685804b4294603928472c11c5 in term 1.
I20260812 14:44:56.185256 11567 raft_consensus.cc:2454] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04739a9685804b4294603928472c11c5 in term 1.
I20260812 14:44:56.185441 11713 leader_election.cc:302] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04739a9685804b4294603928472c11c5, 6618b01bde624897af3f7130eeeca82f; no voters:
I20260812 14:44:56.185542 12197 raft_consensus.cc:2781] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:44:56.185670 12197 raft_consensus.cc:686] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 1 LEADER]: Becoming Leader. State: Replica: 04739a9685804b4294603928472c11c5, State: Running, Role: LEADER
I20260812 14:44:56.185740 12197 consensus_queue.cc:237] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.186458 11270 catalog_manager.cc:5495] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04739a9685804b4294603928472c11c5 (127.15.227.194). New cstate: current_term: 1 leader_uuid: "04739a9685804b4294603928472c11c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } health_report { overall_health: UNKNOWN } } }
I20260812 14:44:56.195802 12184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.193913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.15.227.194:40359)
0812 14:44:56.193958 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:56.193960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:56.193967 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:44:56.193992 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:56.193992 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:56.194006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:56.194079 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:44:56.194177 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.194582 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.195237 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:56.195403 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:56.195440 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:56.195555 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:44:56.195633 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:44:56.195682 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:56.195732 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 14:44:56.195745 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:44:56.197206 12085 raft_consensus.cc:1261] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Refusing update from remote peer 04739a9685804b4294603928472c11c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:56.197211 11567 raft_consensus.cc:1261] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Refusing update from remote peer 04739a9685804b4294603928472c11c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:44:56.197494 12197 consensus_queue.cc:1035] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:56.197620 12202 consensus_queue.cc:1035] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:44:56.199973 12209 mvcc.cc:205] Tried to move back new op lower bound from 7317691990821744640 to 7317691990777356288. Current Snapshot: MvccSnapshot[applied={T|T < 7317691990821744640}]
I20260812 14:44:56.200537 12207 mvcc.cc:205] Tried to move back new op lower bound from 7317691990821744640 to 7317691990777356288. Current Snapshot: MvccSnapshot[applied={T|T < 7317691990821744640 or (T in {7317691990821744640})}]
I20260812 14:44:56.201704 11270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:60594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:56.201787 11270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:60594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:44:56.202457 11270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Sending DeleteTablet for 3 replicas of tablet 3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:44:56.202756 11526 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a927b75dd404ff38bd6f2041d3cb42c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:56 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:59092
I20260812 14:44:56.202775 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 11360
I20260812 14:44:56.202761 12044 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a927b75dd404ff38bd6f2041d3cb42c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:56 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:44412
I20260812 14:44:56.202841 11682 tablet_replica.cc:330] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: stopping tablet replica
I20260812 14:44:56.202881 11682 raft_consensus.cc:2229] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:44:56.202919 11682 raft_consensus.cc:2258] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:56.214561 11798 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a927b75dd404ff38bd6f2041d3cb42c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:44:56 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:35042
I20260812 14:44:56.222824 12230 tablet_replica.cc:330] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: stopping tablet replica
I20260812 14:44:56.222980 12230 raft_consensus.cc:2229] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:44:56.223120 12230 raft_consensus.cc:2258] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:56.223968 12232 tablet_replica.cc:330] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: stopping tablet replica
I20260812 14:44:56.224080 12232 raft_consensus.cc:2229] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:44:56.224182 12232 raft_consensus.cc:2258] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:44:56.224613 12232 ts_tablet_manager.cc:1905] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:56.224742 12230 ts_tablet_manager.cc:1905] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:44:56.226107 12230 ts_tablet_manager.cc:1918] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:56.226192 12230 log.cc:1196] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:44:56.226138 12232 ts_tablet_manager.cc:1918] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:44:56.226373 12232 log.cc:1196] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:44:56.226531 12230 ts_tablet_manager.cc:1939] T 3a927b75dd404ff38bd6f2041d3cb42c P 6618b01bde624897af3f7130eeeca82f: Deleting consensus metadata
I20260812 14:44:56.226792 12232 ts_tablet_manager.cc:1939] T 3a927b75dd404ff38bd6f2041d3cb42c P 04739a9685804b4294603928472c11c5: Deleting consensus metadata
I20260812 14:44:56.226996 11244 catalog_manager.cc:4841] TS 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255): tablet 3a927b75dd404ff38bd6f2041d3cb42c (table test-table [id=6afe84959b5b49e49592c7c27eb7311d]) successfully deleted
I20260812 14:44:56.227468 11249 catalog_manager.cc:4841] TS 04739a9685804b4294603928472c11c5 (127.15.227.194:40359): tablet 3a927b75dd404ff38bd6f2041d3cb42c (table test-table [id=6afe84959b5b49e49592c7c27eb7311d]) successfully deleted
W20260812 14:44:56.229911 11242 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:42391 (error 108)
I20260812 14:44:56.230434 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 11674
I20260812 14:44:56.230460 12187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.230318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:42391 (local address 127.0.0.1:59108)
0812 14:44:56.230378 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.230425 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42391: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 14:44:56.236465 11242 catalog_manager.cc:4584] TS 7ae3bbaa86d94e7daf13978dbe5fced7 (127.15.227.193:42391): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3a927b75dd404ff38bd6f2041d3cb42c: Network error: Client connection negotiation failed: client connection to 127.15.227.193:42391: connect: Connection refused (error 111)
I20260812 14:44:56.245726 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 11920
I20260812 14:44:56.255388 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 11214
I20260812 14:44:56.269338 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45731
--webserver_interface=127.15.227.254
--webserver_port=37915
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:45731
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:56.347358 12238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:56.347536 12238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:56.347568 12238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:56.348647 12238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:44:56.348685 12238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:56.348704 12238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:56.348719 12238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:44:56.348737 12238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:44:56.350221 12238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:45731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:45731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=37915
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:56.350449 12238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:56.350641 12238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:56.355041 12247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.355111 12238 server_base.cc:1034] running on GCE node
W20260812 14:44:56.355016 12251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:56.355038 12248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.355372 12238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:56.355605 12238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:56.356757 12238 hybrid_clock.cc:648] HybridClock initialized: now 1786545896356746 us; error 27 us; skew 500 ppm
Aug 12 14:44:56 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:56.359171 12238 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:44:56.359553 12238 webserver.cc:460] Webserver started at http://127.15.227.254:37915/ using document root <none> and password file <none>
I20260812 14:44:56.359747 12238 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:56.359793 12238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:56.360985 12238 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:56.361589 12262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.361766 12238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:56.361855 12238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "7e0b6218bf604053bff2ac2006057e7c"
format_stamp: "Formatted at 2026-08-12 14:44:54 on dist-test-slave-q9h9"
server_key: "466c20f7bf64b63013edb3c383fd02f5"
server_key_iv: "8252f38dbc75371df4d1715f08dbff98"
server_key_version: "encryptionkey@0"
I20260812 14:44:56.362174 12238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:56.370352 12238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:56.372241 12238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:56.372364 12238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:56.376071 12238 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:45731
I20260812 14:44:56.376101 12350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:45731 every 8 connection(s)
I20260812 14:44:56.376418 12238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:44:56.377784 12351 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:44:56.378419 12351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Bootstrap starting.
I20260812 14:44:56.383868 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 12238
I20260812 14:44:56.384126 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:42391
--local_ip_for_outbound_sockets=127.15.227.193
--tserver_master_addrs=127.15.227.254:45731
--webserver_port=37531
--webserver_interface=127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:56.387038 12351 log.cc:824] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:56.388984 12351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:44:56.389238 12351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c: Bootstrap complete.
I20260812 14:44:56.391223 12351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:56.391424 12351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e0b6218bf604053bff2ac2006057e7c, State: Initialized, Role: FOLLOWER
I20260812 14:44:56.391553 12351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:56.391609 12351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:44:56.391646 12351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:44:56.391691 12351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 2 FOLLOWER]: Advancing to term 3
I20260812 14:44:56.392572 12351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:56.392654 12351 leader_election.cc:302] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e0b6218bf604053bff2ac2006057e7c; no voters:
I20260812 14:44:56.392793 12351 leader_election.cc:288] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 14:44:56.392845 12359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 14:44:56.392956 12359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [term 3 LEADER]: Becoming Leader. State: Replica: 7e0b6218bf604053bff2ac2006057e7c, State: Running, Role: LEADER
I20260812 14:44:56.393041 12351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:44:56.393045 12359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } }
I20260812 14:44:56.393801 12362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e0b6218bf604053bff2ac2006057e7c. Latest consensus state: current_term: 3 leader_uuid: "7e0b6218bf604053bff2ac2006057e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } } }
I20260812 14:44:56.393882 12362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:56.394011 12361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7e0b6218bf604053bff2ac2006057e7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e0b6218bf604053bff2ac2006057e7c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 45731 } } }
I20260812 14:44:56.394090 12361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e0b6218bf604053bff2ac2006057e7c [sys.catalog]: This master's current role is: LEADER
I20260812 14:44:56.394066 12378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:44:56.394923 12378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:44:56.395072 12378 catalog_manager.cc:1259] Loaded cluster ID: f22fa0b023d149fd83e429027ae4bc7f
I20260812 14:44:56.395252 12378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 14:44:56.395534 12378 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fcc170ea9ad google::LogMessage::Fail() at ??:0
@ 0x7fcc170ed88a google::LogMessage::SendToLog() at ??:0
@ 0x7fcc170ea4de google::LogMessage::Flush() at ??:0
@ 0x7fcc170eba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fcc1f00b0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fcc1f0148b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fcc18fb80d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fcc18fad954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fcc19e116db start_thread at ??:0
@ 0x7fcc15be861f clone at ??:0
W20260812 14:44:56.478156 12355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:56.478335 12355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:56.478358 12355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:56.479460 12355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:56.479497 12355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:56.479522 12355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:44:56.480912 12355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:42391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=37531
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:56.481117 12355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:56.481319 12355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:56.486294 12433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.486301 12355 server_base.cc:1034] running on GCE node
W20260812 14:44:56.486279 12434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:56.486507 12436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.486744 12355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:56.486975 12355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:56.488119 12355 hybrid_clock.cc:648] HybridClock initialized: now 1786545896488109 us; error 25 us; skew 500 ppm
Aug 12 14:44:56 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:56.490523 12355 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:44:56.490892 12355 webserver.cc:460] Webserver started at http://127.15.227.193:37531/ using document root <none> and password file <none>
I20260812 14:44:56.491076 12355 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:56.491123 12355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:56.492388 12355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:56.493052 12450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.493209 12355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:56.493294 12355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "7ae3bbaa86d94e7daf13978dbe5fced7"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "3ac143652cb89a0fe812bcd75dcdfcdb"
server_key_iv: "82a793b46ac0a28fb92f0a3647cc81b0"
server_key_version: "encryptionkey@0"
I20260812 14:44:56.493577 12355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:56.529757 12355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:56.531754 12355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:56.531888 12355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:56.532099 12355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:56.532507 12466 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:44:56.534439 12355 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:44:56.534488 12355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:44:56.534520 12355 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:44:56.535156 12355 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:44:56.535188 12355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:44:56.535279 12466 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap starting.
I20260812 14:44:56.541730 12355 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:42391
I20260812 14:44:56.541858 12659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:42391 every 8 connection(s)
I20260812 14:44:56.542160 12355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:44:56.546414 12466 log.cc:824] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:44:56.549067 12466 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:44:56.549412 12466 tablet_bootstrap.cc:492] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Bootstrap complete.
I20260812 14:44:56.549823 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 12355
I20260812 14:44:56.550096 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:40359
--local_ip_for_outbound_sockets=127.15.227.194
--tserver_master_addrs=127.15.227.254:45731
--webserver_port=45943
--webserver_interface=127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:44:56.550622 12466 ts_tablet_manager.cc:1397] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.010s
I20260812 14:44:56.552136 12466 raft_consensus.cc:348] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.552488 12466 raft_consensus.cc:729] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ae3bbaa86d94e7daf13978dbe5fced7, State: Initialized, Role: FOLLOWER
I20260812 14:44:56.552654 12466 consensus_queue.cc:260] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:56.552940 12466 ts_tablet_manager.cc:1428] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260812 14:44:56.623981 12669 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:44:56.542754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:51235)
0812 14:44:56.542922 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.542960 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:56.543366 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:56.623820 (+ 80454us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
W20260812 14:44:56.624239 12662 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:45731 (0 consecutive failures): Network error: Failed to ping master at 127.15.227.254:45731: Client connection negotiation failed: client connection to 127.15.227.254:45731: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:44:56.624615 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.624501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:55515)
0812 14:44:56.624550 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.624591 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260812 14:44:56.624970 12669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.624886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:37557)
0812 14:44:56.624927 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.624952 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 14:44:56.625087 12662 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:44:56.632858 12671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:56.633030 12671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:56.633052 12671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:56.634169 12671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:56.634208 12671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:56.634233 12671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:44:56.635646 12671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:40359
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=45943
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:56.635823 12671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:56.635984 12671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:56.640458 12693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:56.640441 12680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.640630 12671 server_base.cc:1034] running on GCE node
W20260812 14:44:56.640718 12683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.640928 12671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:56.641176 12671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:56.642321 12671 hybrid_clock.cc:648] HybridClock initialized: now 1786545896642314 us; error 34 us; skew 500 ppm
Aug 12 14:44:56 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:56.644558 12671 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:44:56.644959 12671 webserver.cc:460] Webserver started at http://127.15.227.194:45943/ using document root <none> and password file <none>
I20260812 14:44:56.645160 12671 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:56.645207 12671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:56.646422 12671 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:56.646937 12709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.647075 12671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:44:56.647169 12671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "04739a9685804b4294603928472c11c5"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "b21a6a272972bb3831677ac98a6382f9"
server_key_iv: "261383a94ca33bdb5f8967b74c6f225b"
server_key_version: "encryptionkey@0"
I20260812 14:44:56.647441 12671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:56.664209 12671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:56.666134 12671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:56.666260 12671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:56.666445 12671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:56.666710 12671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:56.666742 12671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.666769 12671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:56.666785 12671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.672158 12671 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:40359
I20260812 14:44:56.672194 12927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:40359 every 8 connection(s)
I20260812 14:44:56.672479 12671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:44:56.673348 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.673014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:55541)
0812 14:44:56.673156 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.673221 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
W20260812 14:44:56.673655 12928 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:45731 (0 consecutive failures): Network error: Failed to ping master at 127.15.227.254:45731: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
I20260812 14:44:56.673866 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.673801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:59811)
0812 14:44:56.673827 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.673850 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260812 14:44:56.674073 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.674026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:57111)
0812 14:44:56.674045 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.674062 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260812 14:44:56.674154 12928 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:44:56.679112 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 12671
I20260812 14:44:56.679347 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:35255
--local_ip_for_outbound_sockets=127.15.227.195
--tserver_master_addrs=127.15.227.254:45731
--webserver_port=32969
--webserver_interface=127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:44:56.758729 12933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:44:56.758946 12933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:44:56.758970 12933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:44:56.760061 12933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:44:56.760099 12933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:44:56.760126 12933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:44:56.761514 12933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:35255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=32969
--tserver_master_addrs=127.15.227.254:45731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:44:56.761710 12933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:44:56.761909 12933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:44:56.766338 12946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:44:56.766359 12945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.766403 12933 server_base.cc:1034] running on GCE node
W20260812 14:44:56.766345 12949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:44:56.766726 12933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:44:56.766943 12933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:44:56.768062 12933 hybrid_clock.cc:648] HybridClock initialized: now 1786545896768058 us; error 46 us; skew 500 ppm
Aug 12 14:44:56 dist-test-slave-q9h9 krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:44:56.770324 12933 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:44:56.770712 12933 webserver.cc:460] Webserver started at http://127.15.227.195:32969/ using document root <none> and password file <none>
I20260812 14:44:56.770908 12933 fs_manager.cc:359] Metadata directory not provided
I20260812 14:44:56.770954 12933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:44:56.772116 12933 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:44:56.772629 12965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.772778 12933 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.772852 12933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "6618b01bde624897af3f7130eeeca82f"
format_stamp: "Formatted at 2026-08-12 14:44:55 on dist-test-slave-q9h9"
server_key: "e3fa29ea6760abf225386bbc1c929297"
server_key_iv: "47980e5b27df17da75c21f5d3821950b"
server_key_version: "encryptionkey@0"
I20260812 14:44:56.773113 12933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:44:56.789718 12933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:44:56.791286 12933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:44:56.791406 12933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:44:56.791589 12933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:44:56.791854 12933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:44:56.791893 12933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.791917 12933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:44:56.791942 12933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:44:56.797441 12933 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35255
I20260812 14:44:56.797514 13172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35255 every 8 connection(s)
I20260812 14:44:56.797781 12933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:44:56.798694 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.798379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:55269)
0812 14:44:56.798504 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.798569 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
W20260812 14:44:56.799041 13177 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:45731 (0 consecutive failures): Network error: Failed to ping master at 127.15.227.254:45731: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
I20260812 14:44:56.799275 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.799197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:54093)
0812 14:44:56.799234 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.799259 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260812 14:44:56.799470 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:56.799414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:51627)
0812 14:44:56.799441 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:56.799458 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260812 14:44:56.799548 13177 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:44:56.804298 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 12933
I20260812 14:44:57.625623 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:57.625421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:58231)
0812 14:44:57.625531 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:57.625562 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":68,"threads_started":1}
I20260812 14:44:57.674609 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:57.674413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:55917)
0812 14:44:57.674525 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:57.674556 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":65,"threads_started":1}
I20260812 14:44:57.800004 13190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:57.799815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:40675)
0812 14:44:57.799926 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:57.799957 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":53,"threads_started":1}
I20260812 14:44:57.887682 13191 raft_consensus.cc:482] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:44:57.887820 13191 raft_consensus.cc:504] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:44:57.888186 13191 leader_election.cc:288] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04739a9685804b4294603928472c11c5 (127.15.227.194:40359), 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255)
Aug 12 14:44:57 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.194@KRBTEST.COM
Aug 12 14:44:57 dist-test-slave-q9h9 krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545896, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.195@KRBTEST.COM
I20260812 14:44:57.897776 13193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:57.888467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:40855 (local address 127.15.227.195:35255)
0812 14:44:57.888648 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:57.888656 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:57.888682 (+ 26us) server_negotiation.cc:407] Connection header received
0812 14:44:57.889110 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:57.889112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:57.889151 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:57.889276 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:57.889989 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.890851 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.891771 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.892147 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.892240 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:57.894773 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:57.894776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:57.894776 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:57.894797 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:57.896273 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896614 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:57.896618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:57.896618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:57.896711 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896980 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:57.896981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:57.896982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:57.897194 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:57.897256 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:57.897548 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 14:44:57.897661 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":83,"threads_started":1}
I20260812 14:44:57.897784 13194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:57.888462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:39319 (local address 127.15.227.194:40359)
0812 14:44:57.888633 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 14:44:57.888645 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:44:57.888677 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:44:57.889134 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:44:57.889136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:44:57.889173 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:44:57.889301 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:44:57.890004 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.890883 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.891782 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.892189 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.892290 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:57.894392 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:44:57.894396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:44:57.894396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:44:57.894406 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:44:57.896173 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896633 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:57.896637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:57.896637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:57.896726 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896974 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:44:57.896975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:44:57.896975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:44:57.897203 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:44:57.897290 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:44:57.897538 (+ 248us) server_negotiation.cc:299] Negotiation successful
0812 14:44:57.897677 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":106,"threads_started":1}
I20260812 14:44:57.897835 13192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:57.888323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35255 (local address 127.15.227.193:40855)
0812 14:44:57.888488 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:57.888498 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:57.888549 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:57.889351 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:57.889354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:57.889695 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:57.889877 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:57.889882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.890941 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.890942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:57.891691 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:57.891694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.892223 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.892224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:57.892353 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:57.892869 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:57.892876 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:57.894666 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:57.896341 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:57.896342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:57.896529 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:57.896796 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:57.896797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:57.896916 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:57.897263 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:57.897271 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:57.897388 (+ 117us) client_negotiation.cc:769] Sending connection context
0812 14:44:57.897483 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 14:44:57.897732 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":153,"thread_start_us":66,"threads_started":1}
I20260812 14:44:57.897976 13187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:57.888320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:40359 (local address 127.15.227.193:39319)
0812 14:44:57.888379 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:57.888392 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:44:57.888461 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:44:57.889351 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:44:57.889353 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:44:57.889675 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:44:57.889877 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:57.889882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.890977 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.890978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:57.891693 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:44:57.891696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:44:57.892267 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:44:57.892268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:44:57.892340 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:44:57.892935 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:44:57.892940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:44:57.894297 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:44:57.896261 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896268 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:57.896270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:57.896543 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:57.896824 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:44:57.896824 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:44:57.896825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:44:57.896913 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:44:57.897280 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:44:57.897281 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:44:57.897385 (+ 104us) client_negotiation.cc:769] Sending connection context
0812 14:44:57.897479 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 14:44:57.897656 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":55}
W20260812 14:44:57.898593 12460 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04739a9685804b4294603928472c11c5 (127.15.227.194:40359): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
W20260812 14:44:57.898667 12455 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:44:57.898710 12455 leader_election.cc:302] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae3bbaa86d94e7daf13978dbe5fced7; no voters: 04739a9685804b4294603928472c11c5, 6618b01bde624897af3f7130eeeca82f
I20260812 14:44:57.898806 13191 raft_consensus.cc:2726] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:44:58.626223 13195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:58.625998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:44161)
0812 14:44:58.626136 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:58.626179 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260812 14:44:58.675127 13196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:58.674933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:54709)
0812 14:44:58.675049 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:58.675083 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
I20260812 14:44:58.800549 13198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:58.800361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:35673)
0812 14:44:58.800483 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:58.800515 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260812 14:44:59.626839 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:59.626599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:53297)
0812 14:44:59.626744 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:59.626785 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260812 14:44:59.675666 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:59.675465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:50661)
0812 14:44:59.675587 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:59.675623 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260812 14:44:59.801100 13202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:44:59.800904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:42233)
0812 14:44:59.801027 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:44:59.801062 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":76,"threads_started":1}
I20260812 14:45:00.120064 13204 raft_consensus.cc:482] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:00.120154 13204 raft_consensus.cc:504] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:45:00.120323 13204 leader_election.cc:288] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04739a9685804b4294603928472c11c5 (127.15.227.194:40359), 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255)
W20260812 14:45:00.120682 12455 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
W20260812 14:45:00.120761 12460 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04739a9685804b4294603928472c11c5 (127.15.227.194:40359): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:45:00.120841 12460 leader_election.cc:302] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae3bbaa86d94e7daf13978dbe5fced7; no voters: 04739a9685804b4294603928472c11c5, 6618b01bde624897af3f7130eeeca82f
I20260812 14:45:00.120925 13204 raft_consensus.cc:2726] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:45:00.627394 13205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:00.627198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:34301)
0812 14:45:00.627313 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:00.627347 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260812 14:45:00.676213 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:00.676009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:57717)
0812 14:45:00.676115 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:00.676161 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260812 14:45:00.801669 13208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:00.801472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:40715)
0812 14:45:00.801594 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:00.801628 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260812 14:45:01.628093 13212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:01.627866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:37471)
0812 14:45:01.628000 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:01.628036 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260812 14:45:01.676759 13213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:01.676587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:55727)
0812 14:45:01.676689 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:01.676717 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
I20260812 14:45:01.802217 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:01.802008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:47727)
0812 14:45:01.802147 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:01.802179 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
I20260812 14:45:02.628650 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:02.628468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:38817)
0812 14:45:02.628576 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:02.628606 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":63,"threads_started":1}
I20260812 14:45:02.659492 13217 raft_consensus.cc:482] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:02.659569 13217 raft_consensus.cc:504] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:45:02.659724 13217 leader_election.cc:288] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04739a9685804b4294603928472c11c5 (127.15.227.194:40359), 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255)
W20260812 14:45:02.660071 12460 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04739a9685804b4294603928472c11c5 (127.15.227.194:40359): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
W20260812 14:45:02.660153 12455 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:45:02.660216 12455 leader_election.cc:302] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae3bbaa86d94e7daf13978dbe5fced7; no voters: 04739a9685804b4294603928472c11c5, 6618b01bde624897af3f7130eeeca82f
I20260812 14:45:02.660310 13217 raft_consensus.cc:2726] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:45:02.677237 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:02.677069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:34845)
0812 14:45:02.677172 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:02.677202 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
I20260812 14:45:02.802788 13222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:02.802582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:45287)
0812 14:45:02.802713 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:02.802750 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260812 14:45:03.629254 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:03.629015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:50145)
0812 14:45:03.629142 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:03.629182 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260812 14:45:03.677752 13224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:03.677558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:54685)
0812 14:45:03.677679 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:03.677707 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260812 14:45:03.803306 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:03.803134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:50755)
0812 14:45:03.803237 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:03.803269 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
I20260812 14:45:04.629911 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:04.629695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:56683)
0812 14:45:04.629823 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:04.629860 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260812 14:45:04.678274 13233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:04.678085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:40469)
0812 14:45:04.678200 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:04.678234 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260812 14:45:04.803851 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:04.803650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:54059)
0812 14:45:04.803774 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:04.803809 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260812 14:45:05.177441 13236 raft_consensus.cc:482] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:05.177528 13236 raft_consensus.cc:504] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:45:05.177726 13236 leader_election.cc:288] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04739a9685804b4294603928472c11c5 (127.15.227.194:40359), 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255)
W20260812 14:45:05.178068 12460 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04739a9685804b4294603928472c11c5 (127.15.227.194:40359): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
W20260812 14:45:05.178157 12455 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:45:05.178218 12455 leader_election.cc:302] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae3bbaa86d94e7daf13978dbe5fced7; no voters: 04739a9685804b4294603928472c11c5, 6618b01bde624897af3f7130eeeca82f
I20260812 14:45:05.178295 13236 raft_consensus.cc:2726] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:45:05.630548 13237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:05.630333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:53759)
0812 14:45:05.630466 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:05.630497 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":94,"threads_started":1}
I20260812 14:45:05.678841 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:05.678656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:60161)
0812 14:45:05.678770 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:05.678802 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260812 14:45:05.804418 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:05.804238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:41699)
0812 14:45:05.804346 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:05.804379 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260812 14:45:06.631156 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:06.630968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:33689)
0812 14:45:06.631078 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:06.631111 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260812 14:45:06.679358 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:06.679179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:59331)
0812 14:45:06.679287 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:06.679320 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
I20260812 14:45:06.804958 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:06.804782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:42377)
0812 14:45:06.804888 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:06.804921 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260812 14:45:07.631783 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:07.631562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:34565)
0812 14:45:07.631685 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:07.631725 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260812 14:45:07.679920 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:07.679717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:49495)
0812 14:45:07.679853 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:07.679884 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260812 14:45:07.805506 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:07.805308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:36945)
0812 14:45:07.805434 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:07.805469 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260812 14:45:08.198201 13256 raft_consensus.cc:482] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:08.198303 13256 raft_consensus.cc:504] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04739a9685804b4294603928472c11c5" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 40359 } } peers { permanent_uuid: "6618b01bde624897af3f7130eeeca82f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35255 } } peers { permanent_uuid: "7ae3bbaa86d94e7daf13978dbe5fced7" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 42391 } }
I20260812 14:45:08.198485 13256 leader_election.cc:288] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 04739a9685804b4294603928472c11c5 (127.15.227.194:40359), 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255)
W20260812 14:45:08.198817 12455 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6618b01bde624897af3f7130eeeca82f (127.15.227.195:35255): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
W20260812 14:45:08.198907 12460 leader_election.cc:341] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 04739a9685804b4294603928472c11c5 (127.15.227.194:40359): Not found: Tablet not found: 3a927b75dd404ff38bd6f2041d3cb42c
I20260812 14:45:08.198979 12460 leader_election.cc:302] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ae3bbaa86d94e7daf13978dbe5fced7; no voters: 04739a9685804b4294603928472c11c5, 6618b01bde624897af3f7130eeeca82f
I20260812 14:45:08.199067 13256 raft_consensus.cc:2726] T 3a927b75dd404ff38bd6f2041d3cb42c P 7ae3bbaa86d94e7daf13978dbe5fced7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:45:08.632437 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:08.632243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:44779)
0812 14:45:08.632357 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:08.632391 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260812 14:45:08.680485 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:08.680282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:51059)
0812 14:45:08.680387 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:08.680419 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260812 14:45:08.806044 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:08.805846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:52383)
0812 14:45:08.805957 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:08.805987 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
I20260812 14:45:09.632994 13262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:09.632799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:42579)
0812 14:45:09.632915 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:09.632950 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260812 14:45:09.680958 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:09.680797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:56851)
0812 14:45:09.680895 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:09.680922 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":57,"threads_started":1}
I20260812 14:45:09.806573 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:09.806402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:32903)
0812 14:45:09.806504 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:09.806533 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":59,"threads_started":1}
I20260812 14:45:10.633543 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:10.633346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:42605)
0812 14:45:10.633463 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:10.633497 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260812 14:45:10.681471 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:10.681292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:40277)
0812 14:45:10.681397 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:10.681429 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260812 14:45:10.807086 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:10.806908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:32825)
0812 14:45:10.807017 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:10.807048 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260812 14:45:11.634186 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:11.633973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.193:53235)
0812 14:45:11.634106 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:11.634141 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260812 14:45:11.682005 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:11.681826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.194:59779)
0812 14:45:11.681937 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:11.681968 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260812 14:45:11.807664 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:11.807468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:45731 (local address 127.15.227.195:40961)
0812 14:45:11.807589 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:11.807624 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:45731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260812 14:45:11.812057 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 12355
I20260812 14:45:11.824865 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 12671
I20260812 14:45:11.835809 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 12933
2026-08-12T14:45:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18403 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:45:11 dist-test-slave-q9h9 krb5kdc[13280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:45:11 dist-test-slave-q9h9 krb5kdc[13280](info): set up 2 sockets
Aug 12 14:45:11 dist-test-slave-q9h9 krb5kdc[13280](info): commencing operation
krb5kdc: starting...
W20260812 14:45:13.041216 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:45:13.148442 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33275
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:45:13.227226 13310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:13.227409 13310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:13.227430 13310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:13.228483 13310 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:45:13.228513 13310 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:45:13.228525 13310 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:45:13.228535 13310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:45:13.228549 13310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:13.228565 13310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:13.228579 13310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:45:13.228590 13310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:45:13.229972 13310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:13.230186 13310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:13.230348 13310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:13.234966 13321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:13.235090 13310 server_base.cc:1034] running on GCE node
W20260812 14:45:13.235080 13323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:13.235028 13331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:13.235464 13310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:13.235697 13310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:13.236836 13310 hybrid_clock.cc:648] HybridClock initialized: now 1786545913236825 us; error 32 us; skew 500 ppm
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:13.239135 13310 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:45:13.239504 13310 webserver.cc:460] Webserver started at http://127.15.227.254:41527/ using document root <none> and password file <none>
I20260812 14:45:13.239682 13310 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:13.239722 13310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:13.239805 13310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:45:13.240693 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "2b11de877d554cc39cb809d4550b9fe6"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "3aef6dd2f109c24ceca7291bfb4c1a0e"
server_key_iv: "3b331564cc140d062a029d069d544dbf"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.240963 13310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "2b11de877d554cc39cb809d4550b9fe6"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "3aef6dd2f109c24ceca7291bfb4c1a0e"
server_key_iv: "3b331564cc140d062a029d069d544dbf"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.242101 13310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:45:13.242739 13344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.242892 13310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:13.242969 13310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "2b11de877d554cc39cb809d4550b9fe6"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "3aef6dd2f109c24ceca7291bfb4c1a0e"
server_key_iv: "3b331564cc140d062a029d069d544dbf"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.243018 13310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:13.253010 13310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:13.254890 13310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:13.254990 13310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:13.258543 13310 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:33275
I20260812 14:45:13.258574 13440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:33275 every 8 connection(s)
I20260812 14:45:13.258854 13310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:45:13.259603 13441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:45:13.261564 13441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Bootstrap starting.
I20260812 14:45:13.262125 13441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Neither blocks nor log segments found. Creating new log.
I20260812 14:45:13.262424 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 13310
I20260812 14:45:13.262435 13441 log.cc:824] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:13.262524 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:45:13.262732 16271 external_mini_cluster.cc:1428] Setting key 10c547f8db23e866c68d0331d1663024
I20260812 14:45:13.263110 13441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: No bootstrap required, opened a new log
I20260812 14:45:13.264842 13441 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:13.264961 13441 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:45:13.264981 13441 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b11de877d554cc39cb809d4550b9fe6, State: Initialized, Role: FOLLOWER
I20260812 14:45:13.265093 13441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:13.265151 13441 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:45:13.265170 13441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:45:13.265200 13441 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:45:13.265692 13441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:13.265776 13441 leader_election.cc:302] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b11de877d554cc39cb809d4550b9fe6; no voters:
I20260812 14:45:13.265916 13441 leader_election.cc:288] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:45:13.265961 13450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:45:13.266206 13441 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:45:13.266251 13450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 LEADER]: Becoming Leader. State: Replica: 2b11de877d554cc39cb809d4550b9fe6, State: Running, Role: LEADER
I20260812 14:45:13.266350 13450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:13.266676 13450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b11de877d554cc39cb809d4550b9fe6. Latest consensus state: current_term: 1 leader_uuid: "2b11de877d554cc39cb809d4550b9fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } } }
I20260812 14:45:13.266902 13450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:45:13.266839 13452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b11de877d554cc39cb809d4550b9fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } } }
I20260812 14:45:13.267135 13452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:45:13.267501 13466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:45:13.267993 13466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:45:13.269477 13466 catalog_manager.cc:1347] Generated new cluster ID: dcbed9ebcdca4e86a70f16750cd1fa8a
I20260812 14:45:13.269522 13466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:45:13.273916 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.263193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.15.227.254:33275)
0812 14:45:13.263378 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.263385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.263407 (+ 22us) server_negotiation.cc:407] Connection header received
0812 14:45:13.264187 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.264194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.264255 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.264381 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:13.264953 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.266357 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.267400 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.267967 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.268078 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.270637 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:13.270641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:13.270641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:13.270652 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:13.272452 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.272815 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.272818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.272818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.272919 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.273089 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.273089 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.273090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.273380 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:13.273521 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.273586 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.273714 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260812 14:45:13.285436 13466 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:45:13.286075 13466 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:45:13.287691 13466 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Generated new TSK 0
I20260812 14:45:13.287829 13466 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:45:13.310211 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33275
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:45:13.389389 13504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:13.389572 13504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:13.389595 13504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:13.390686 13504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:13.390719 13504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:13.390744 13504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:45:13.392098 13504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:13.392273 13504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:13.392443 13504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:13.396957 13517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:13.396963 13515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:13.397135 13504 server_base.cc:1034] running on GCE node
W20260812 14:45:13.396967 13514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:13.397361 13504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:13.397557 13504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:13.398681 13504 hybrid_clock.cc:648] HybridClock initialized: now 1786545913398662 us; error 36 us; skew 500 ppm
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:13.400949 13504 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:45:13.401356 13504 webserver.cc:460] Webserver started at http://127.15.227.193:44815/ using document root <none> and password file <none>
I20260812 14:45:13.401520 13504 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:13.401569 13504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:13.401664 13504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:45:13.402612 13504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "73479b3759e745aebc58100a407dc5fb"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "86b4a4cfd5368ea843c8927095dcb9b2"
server_key_iv: "64782488e502e01a51dd953b35b52fd2"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.402892 13504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "73479b3759e745aebc58100a407dc5fb"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "86b4a4cfd5368ea843c8927095dcb9b2"
server_key_iv: "64782488e502e01a51dd953b35b52fd2"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.404016 13504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:45:13.404647 13531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.404794 13504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:13.404872 13504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "73479b3759e745aebc58100a407dc5fb"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "86b4a4cfd5368ea843c8927095dcb9b2"
server_key_iv: "64782488e502e01a51dd953b35b52fd2"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.404932 13504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:13.420753 13504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:13.422799 13504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:13.422930 13504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:13.423144 13504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:13.423436 13504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:45:13.423468 13504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.423494 13504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:45:13.423518 13504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.429032 13504 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:33809
I20260812 14:45:13.429105 13752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:33809 every 8 connection(s)
I20260812 14:45:13.429361 13504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:45:13.434396 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 13504
I20260812 14:45:13.434484 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:45:13.434691 16271 external_mini_cluster.cc:1428] Setting key ac9e8ee5ff1ca48269e2b85abff69398
I20260812 14:45:13.437633 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.429983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:38907 (local address 127.15.227.254:33275)
0812 14:45:13.430055 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.430057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.430243 (+ 186us) server_negotiation.cc:407] Connection header received
0812 14:45:13.430691 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.430692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.430702 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.430749 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:13.431563 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.432044 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.432913 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.433096 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.433137 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.435071 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:13.435072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:13.435072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:13.435078 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:13.436353 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.436779 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.436780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.436781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.436844 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.437038 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.437039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.437040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.437161 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:13.437228 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.437507 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.437571 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:45:13.437848 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.429962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:38907)
0812 14:45:13.430137 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.430171 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.430606 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.430805 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.430808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.431328 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:13.431501 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.431506 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.432103 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.432103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.432855 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.432857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.433145 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.433146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.433232 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.433590 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:13.433598 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:13.434989 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:13.436431 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:13.436437 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:13.436438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:13.436714 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:13.436902 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:13.436903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:13.436903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:13.436993 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:13.437225 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:13.437230 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:13.437382 (+ 152us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.437489 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.437649 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260812 14:45:13.438308 13755 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33275
I20260812 14:45:13.438400 13755 heartbeater.cc:464] Registering TS with master...
I20260812 14:45:13.438563 13755 heartbeater.cc:511] Master 127.15.227.254:33275 requested a full tablet report, sending...
I20260812 14:45:13.439008 13377 ts_manager.cc:194] Registered new tserver with Master: 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809)
I20260812 14:45:13.439610 13377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:38907
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:45:13.457502 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33275
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:45:13.535583 13767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:13.535758 13767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:13.535779 13767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:13.536916 13767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:13.536957 13767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:13.536983 13767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:45:13.538411 13767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:13.538631 13767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:13.538843 13767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:13.543393 13776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:13.543414 13774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:13.543454 13781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:13.543497 13767 server_base.cc:1034] running on GCE node
I20260812 14:45:13.543716 13767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:13.543942 13767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:13.545083 13767 hybrid_clock.cc:648] HybridClock initialized: now 1786545913545068 us; error 29 us; skew 500 ppm
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:13.547326 13767 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:45:13.547729 13767 webserver.cc:460] Webserver started at http://127.15.227.194:42991/ using document root <none> and password file <none>
I20260812 14:45:13.547932 13767 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:13.547979 13767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:13.548099 13767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:45:13.549017 13767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "1ea6f87083c4487485a473c99bf3d1ef"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "6c2ba386d0d06d1c3436680c6781f360"
server_key_iv: "5a6dd94aafac4626e77035ce5814dd14"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.549324 13767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "1ea6f87083c4487485a473c99bf3d1ef"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "6c2ba386d0d06d1c3436680c6781f360"
server_key_iv: "5a6dd94aafac4626e77035ce5814dd14"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.550545 13767 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:45:13.551224 13795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.551368 13767 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:13.551446 13767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "1ea6f87083c4487485a473c99bf3d1ef"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "6c2ba386d0d06d1c3436680c6781f360"
server_key_iv: "5a6dd94aafac4626e77035ce5814dd14"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.551506 13767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:13.566298 13767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:13.568025 13767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:13.568146 13767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:13.568334 13767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:13.568601 13767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:45:13.568634 13767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.568660 13767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:45:13.568684 13767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.574091 13767 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:39075
I20260812 14:45:13.574151 14005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:39075 every 8 connection(s)
I20260812 14:45:13.574412 13767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:45:13.581425 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 13767
I20260812 14:45:13.581535 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:45:13.581729 16271 external_mini_cluster.cc:1428] Setting key 460189acfafa47361e1c42264dabd94a
I20260812 14:45:13.582233 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.575029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:38167 (local address 127.15.227.254:33275)
0812 14:45:13.575071 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.575073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.575259 (+ 186us) server_negotiation.cc:407] Connection header received
0812 14:45:13.575665 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.575666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.575676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.575722 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:13.576511 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.576999 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.577868 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.578085 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.578140 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.579891 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:13.579892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:13.579892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:13.579899 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:13.580933 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.581241 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.581241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.581241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.581297 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.581569 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.581570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.581570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.581690 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:13.581743 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.582062 (+ 319us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.582146 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:45:13.582369 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.575006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:38167)
0812 14:45:13.575152 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.575183 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.575593 (+ 410us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.575780 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.575783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.576279 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:13.576447 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.576451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.577058 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.577059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.577810 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.577812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.578144 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.578144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.578239 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.578578 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:13.578584 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:13.579813 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:13.580992 (+ 1179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:13.580997 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:13.580999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:13.581188 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:13.581366 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:13.581367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:13.581367 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:13.581505 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:13.581748 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:13.581755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:13.581901 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.582005 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.582165 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260812 14:45:13.582767 14008 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33275
I20260812 14:45:13.582854 14008 heartbeater.cc:464] Registering TS with master...
I20260812 14:45:13.583019 14008 heartbeater.cc:511] Master 127.15.227.254:33275 requested a full tablet report, sending...
I20260812 14:45:13.583379 13377 ts_manager.cc:194] Registered new tserver with Master: 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
I20260812 14:45:13.583725 13377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:38167
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:45:13.602872 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33275
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:45:13.682794 14015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:13.682972 14015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:13.682993 14015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:13.684095 14015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:13.684134 14015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:13.684158 14015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:45:13.685528 14015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:13.685710 14015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:13.685881 14015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:13.690426 14026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:13.690428 14022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:13.690423 14020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:13.690667 14015 server_base.cc:1034] running on GCE node
I20260812 14:45:13.690851 14015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:13.691059 14015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:13.692188 14015 hybrid_clock.cc:648] HybridClock initialized: now 1786545913692175 us; error 27 us; skew 500 ppm
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:13.694413 14015 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:45:13.694845 14015 webserver.cc:460] Webserver started at http://127.15.227.195:35019/ using document root <none> and password file <none>
I20260812 14:45:13.695055 14015 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:13.695102 14015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:13.695225 14015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:45:13.696130 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "ed8e402316f042f090bbfb56846624e0"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "fc07e66e8e26150ce6e54ffd516364a6"
server_key_iv: "3ce8512abb7c505a694699a6c46b1ce4"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.696453 14015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "ed8e402316f042f090bbfb56846624e0"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "fc07e66e8e26150ce6e54ffd516364a6"
server_key_iv: "3ce8512abb7c505a694699a6c46b1ce4"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.697602 14015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:45:13.698264 14042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.698407 14015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.698498 14015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "ed8e402316f042f090bbfb56846624e0"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "fc07e66e8e26150ce6e54ffd516364a6"
server_key_iv: "3ce8512abb7c505a694699a6c46b1ce4"
server_key_version: "encryptionkey@0"
I20260812 14:45:13.698562 14015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:13.707360 14015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:13.708985 14015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:13.709108 14015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:13.709295 14015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:13.709559 14015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:45:13.709590 14015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.709616 14015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:45:13.709641 14015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:13.715031 14015 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:37929
I20260812 14:45:13.715087 14264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:37929 every 8 connection(s)
I20260812 14:45:13.715340 14015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:45:13.716671 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 14015
I20260812 14:45:13.716795 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:45:13.717005 16271 external_mini_cluster.cc:1428] Setting key d62dcc44a40c3f26cccf65d77b494e8c
Aug 12 14:45:13 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545913, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:45:13.723214 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.715926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:60783 (local address 127.15.227.254:33275)
0812 14:45:13.715977 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.715979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.716157 (+ 178us) server_negotiation.cc:407] Connection header received
0812 14:45:13.716620 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.716621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.716632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.716671 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:13.717626 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.718251 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.719119 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.719294 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.719336 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.721056 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:13.721058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:13.721058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:13.721065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:13.722305 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.722594 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.722594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.722595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.722639 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.722802 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.722803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.722803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.722891 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:13.722935 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.723104 (+ 169us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.723154 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:45:13.723363 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.715884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:60783)
0812 14:45:13.716060 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.716089 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.716506 (+ 417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.716747 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.716751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.717359 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:13.717563 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.717568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.718322 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.718323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.719057 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.719059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.719342 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.719343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.719428 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.719776 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:13.719783 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:13.720982 (+ 1199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:13.722359 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:13.722362 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:13.722364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:13.722542 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:13.722679 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:13.722679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:13.722680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:13.722760 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:13.722931 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:13.722933 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:13.723024 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.723092 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.723191 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260812 14:45:13.723784 14267 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33275
I20260812 14:45:13.723858 14267 heartbeater.cc:464] Registering TS with master...
I20260812 14:45:13.724009 14267 heartbeater.cc:511] Master 127.15.227.254:33275 requested a full tablet report, sending...
I20260812 14:45:13.724356 13377 ts_manager.cc:194] Registered new tserver with Master: ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929)
I20260812 14:45:13.724678 13377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:60783
I20260812 14:45:13.728428 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:45:13.734377 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.729584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.15.227.254:33275)
0812 14:45:13.729641 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.729642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.729723 (+ 81us) server_negotiation.cc:407] Connection header received
0812 14:45:13.729774 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.729775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.729782 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.729822 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:13.730340 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.730757 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.731397 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.731555 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.731594 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.732476 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:13.732477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:13.732477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:13.732482 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:13.733567 (+ 1085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.733802 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.733803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.733803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.733845 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:13.733959 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:13.733960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:13.733960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:13.734114 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:13.734183 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.734266 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.734319 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:45:13.735558 13377 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36604:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:45:13.737002 13377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:45:13.742784 14281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.738839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33809 (local address 127.0.0.1:59812)
0812 14:45:13.738980 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.739013 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.739069 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.739513 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.739514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.739516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:13.739560 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.739562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.740627 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.740628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.741787 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.741790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.742559 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.742560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.742611 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.742614 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.742641 (+ 27us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.742698 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":22,"thread_start_us":84,"threads_started":1}
I20260812 14:45:13.742777 14282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.739016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:39075 (local address 127.0.0.1:35752)
0812 14:45:13.739358 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.739367 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.739421 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.739665 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.739666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.739668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:13.739709 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.739710 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.740908 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.740909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.741671 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.741674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.742508 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.742509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.742586 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.742601 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.742666 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.742707 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":8,"thread_start_us":64,"threads_started":1}
I20260812 14:45:13.743023 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.739115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.15.227.194:39075)
0812 14:45:13.739299 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.739303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.739413 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:45:13.739496 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.739499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.739514 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.739580 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:13.739783 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.740830 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.741764 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.742426 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.742497 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.742547 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.742779 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.742823 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260812 14:45:13.743033 14284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.738954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59812 (local address 127.15.227.193:33809)
0812 14:45:13.739144 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.739148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.739159 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:45:13.739207 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.739209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.739224 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.739523 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:13.739666 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.740539 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.741880 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.742468 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.742830 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.742896 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.742923 (+ 27us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.742965 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
I20260812 14:45:13.743451 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.738840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:37929 (local address 127.0.0.1:36992)
0812 14:45:13.739225 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.739233 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.739283 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.739439 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.739442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.739444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:13.739511 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.739513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.740870 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.740871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.741944 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.741946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.742812 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.742813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.743281 (+ 468us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.743286 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.743349 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.743387 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260812 14:45:13.743481 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.738888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36992 (local address 127.15.227.195:37929)
0812 14:45:13.739179 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.739182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.739283 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:45:13.739343 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.739346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.739359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.739432 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:13.739630 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.740775 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.742033 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.742668 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.742730 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.742779 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.743374 (+ 595us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.743415 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":67,"threads_started":1}
I20260812 14:45:13.743683 13612 tablet_service.cc:1467] Processing CreateTablet for tablet 15946ab2a3894ef7a4a719144a674304 (DEFAULT_TABLE table=test-table [id=dfbb5dd323274a008b8acddf4f069835]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:45:13.743746 13887 tablet_service.cc:1467] Processing CreateTablet for tablet 15946ab2a3894ef7a4a719144a674304 (DEFAULT_TABLE table=test-table [id=dfbb5dd323274a008b8acddf4f069835]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:45:13.744033 13612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15946ab2a3894ef7a4a719144a674304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:45:13.744082 13887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15946ab2a3894ef7a4a719144a674304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:45:13.744366 14129 tablet_service.cc:1467] Processing CreateTablet for tablet 15946ab2a3894ef7a4a719144a674304 (DEFAULT_TABLE table=test-table [id=dfbb5dd323274a008b8acddf4f069835]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:45:13.744621 14129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15946ab2a3894ef7a4a719144a674304. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:45:13.746556 14288 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Bootstrap starting.
I20260812 14:45:13.747114 14288 tablet_bootstrap.cc:654] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Neither blocks nor log segments found. Creating new log.
I20260812 14:45:13.747380 14290 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Bootstrap starting.
I20260812 14:45:13.747620 14288 log.cc:824] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:13.747990 14290 tablet_bootstrap.cc:654] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Neither blocks nor log segments found. Creating new log.
I20260812 14:45:13.748507 14290 log.cc:824] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:13.748821 14291 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Bootstrap starting.
I20260812 14:45:13.749048 14288 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: No bootstrap required, opened a new log
I20260812 14:45:13.749125 14290 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: No bootstrap required, opened a new log
I20260812 14:45:13.749185 14290 ts_tablet_manager.cc:1397] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:45:13.749271 14288 ts_tablet_manager.cc:1397] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260812 14:45:13.750735 14290 raft_consensus.cc:348] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.750885 14290 raft_consensus.cc:374] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:45:13.750905 14290 raft_consensus.cc:729] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73479b3759e745aebc58100a407dc5fb, State: Initialized, Role: FOLLOWER
I20260812 14:45:13.751013 14290 consensus_queue.cc:260] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.751250 14290 ts_tablet_manager.cc:1428] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:45:13.751238 14288 raft_consensus.cc:348] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.751364 13755 heartbeater.cc:503] Master 127.15.227.254:33275 was elected leader, sending a full tablet report...
I20260812 14:45:13.751390 14288 raft_consensus.cc:374] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:45:13.751433 14288 raft_consensus.cc:729] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea6f87083c4487485a473c99bf3d1ef, State: Initialized, Role: FOLLOWER
I20260812 14:45:13.751541 14288 consensus_queue.cc:260] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.751773 14288 ts_tablet_manager.cc:1428] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:45:13.751886 14008 heartbeater.cc:503] Master 127.15.227.254:33275 was elected leader, sending a full tablet report...
I20260812 14:45:13.751906 14291 tablet_bootstrap.cc:654] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Neither blocks nor log segments found. Creating new log.
I20260812 14:45:13.752424 14291 log.cc:824] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:13.753085 14291 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: No bootstrap required, opened a new log
I20260812 14:45:13.753142 14291 ts_tablet_manager.cc:1397] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:45:13.754585 14291 raft_consensus.cc:348] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.754715 14291 raft_consensus.cc:374] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:45:13.754755 14291 raft_consensus.cc:729] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed8e402316f042f090bbfb56846624e0, State: Initialized, Role: FOLLOWER
I20260812 14:45:13.754879 14291 consensus_queue.cc:260] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.755122 14291 ts_tablet_manager.cc:1428] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:45:13.755156 14267 heartbeater.cc:503] Master 127.15.227.254:33275 was elected leader, sending a full tablet report...
W20260812 14:45:13.824899 14009 tablet.cc:2369] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:45:13.885352 14303 raft_consensus.cc:482] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:13.885480 14303 raft_consensus.cc:504] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.885825 14303 leader_election.cc:288] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809), 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
I20260812 14:45:13.888468 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.886065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:59875 (local address 127.15.227.194:39075)
0812 14:45:13.886175 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.886178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.886185 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:45:13.886207 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.886207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.886214 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.886255 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:13.886396 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.886866 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.887796 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.888167 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.888227 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.888235 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.888368 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.888401 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 14:45:13.888476 14269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.885966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:39075 (local address 127.15.227.195:59875)
0812 14:45:13.886048 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.886061 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.886101 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.886280 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.886281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.886283 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:13.886340 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.886341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.886934 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.886935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.887715 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.887718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.888236 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.888237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.888311 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.888313 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.888357 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.888396 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:45:13.888470 14284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.886065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:34433 (local address 127.15.227.193:33809)
0812 14:45:13.886113 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.886115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.886216 (+ 101us) server_negotiation.cc:407] Connection header received
0812 14:45:13.886275 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.886276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.886285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.886330 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:13.886471 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.886918 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.887802 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.888166 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.888227 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.888235 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.888371 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.888407 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:45:13.888494 14305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.885967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33809 (local address 127.15.227.195:34433)
0812 14:45:13.886147 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.886158 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:13.886216 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:13.886342 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:13.886343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:13.886345 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:13.886411 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.886412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.886991 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.886992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.887716 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:13.887718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.888241 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.888242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:13.888307 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.888311 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:45:13.888356 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:45:13.888396 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":10,"thread_start_us":110,"threads_started":1}
I20260812 14:45:13.888850 13922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15946ab2a3894ef7a4a719144a674304" candidate_uuid: "ed8e402316f042f090bbfb56846624e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ea6f87083c4487485a473c99bf3d1ef" is_pre_election: true
I20260812 14:45:13.888989 13922 raft_consensus.cc:2454] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed8e402316f042f090bbfb56846624e0 in term 0.
I20260812 14:45:13.889048 13662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15946ab2a3894ef7a4a719144a674304" candidate_uuid: "ed8e402316f042f090bbfb56846624e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73479b3759e745aebc58100a407dc5fb" is_pre_election: true
I20260812 14:45:13.889173 13662 raft_consensus.cc:2454] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed8e402316f042f090bbfb56846624e0 in term 0.
I20260812 14:45:13.889218 14044 leader_election.cc:302] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ea6f87083c4487485a473c99bf3d1ef, ed8e402316f042f090bbfb56846624e0; no voters:
I20260812 14:45:13.889400 14303 raft_consensus.cc:2781] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:45:13.889449 14303 raft_consensus.cc:482] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:45:13.889464 14303 raft_consensus.cc:3037] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:45:13.890120 14303 raft_consensus.cc:504] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.890223 14303 leader_election.cc:288] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [CANDIDATE]: Term 1 election: Requested vote from peers 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809), 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
I20260812 14:45:13.890370 13662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15946ab2a3894ef7a4a719144a674304" candidate_uuid: "ed8e402316f042f090bbfb56846624e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73479b3759e745aebc58100a407dc5fb"
I20260812 14:45:13.890403 13922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15946ab2a3894ef7a4a719144a674304" candidate_uuid: "ed8e402316f042f090bbfb56846624e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ea6f87083c4487485a473c99bf3d1ef"
I20260812 14:45:13.890463 13922 raft_consensus.cc:3037] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:45:13.890458 13662 raft_consensus.cc:3037] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:45:13.891129 13922 raft_consensus.cc:2454] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed8e402316f042f090bbfb56846624e0 in term 1.
I20260812 14:45:13.891145 13662 raft_consensus.cc:2454] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed8e402316f042f090bbfb56846624e0 in term 1.
I20260812 14:45:13.891305 14049 leader_election.cc:302] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73479b3759e745aebc58100a407dc5fb, ed8e402316f042f090bbfb56846624e0; no voters:
I20260812 14:45:13.891377 14303 raft_consensus.cc:2781] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:45:13.891512 14303 raft_consensus.cc:686] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 1 LEADER]: Becoming Leader. State: Replica: ed8e402316f042f090bbfb56846624e0, State: Running, Role: LEADER
I20260812 14:45:13.891594 14303 consensus_queue.cc:237] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:13.892346 13375 catalog_manager.cc:5495] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed8e402316f042f090bbfb56846624e0 (127.15.227.195). New cstate: current_term: 1 leader_uuid: "ed8e402316f042f090bbfb56846624e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260812 14:45:13.923233 14283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.921380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36996 (local address 127.15.227.195:37929)
0812 14:45:13.921416 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:13.921418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:13.921426 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:45:13.921468 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:13.921469 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:13.921483 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:13.921530 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:45:13.921622 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.922061 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.922697 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:13.922870 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:13.922910 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:13.923010 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:45:13.923095 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:45:13.923145 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:13.923160 (+ 15us) server_negotiation.cc:299] Negotiation successful
0812 14:45:13.923173 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:45:13.924721 13662 raft_consensus.cc:1261] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Refusing update from remote peer ed8e402316f042f090bbfb56846624e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:45:13.924721 13922 raft_consensus.cc:1261] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 FOLLOWER]: Refusing update from remote peer ed8e402316f042f090bbfb56846624e0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:45:13.925050 14308 consensus_queue.cc:1035] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:45:13.925172 14303 consensus_queue.cc:1035] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:45:13.929771 13377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:45:13.929855 13377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36604:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:45:13.930517 13377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Sending DeleteTablet for 3 replicas of tablet 15946ab2a3894ef7a4a719144a674304
I20260812 14:45:13.930828 13612 tablet_service.cc:1514] Processing DeleteTablet for tablet 15946ab2a3894ef7a4a719144a674304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:45:13 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:59812
I20260812 14:45:13.930867 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 13504
I20260812 14:45:13.930866 13887 tablet_service.cc:1514] Processing DeleteTablet for tablet 15946ab2a3894ef7a4a719144a674304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:45:13 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:35752
I20260812 14:45:13.931034 14331 tablet_replica.cc:330] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: stopping tablet replica
I20260812 14:45:13.931102 14331 raft_consensus.cc:2229] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:45:13.931171 14331 raft_consensus.cc:2258] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:45:13.940119 14129 tablet_service.cc:1514] Processing DeleteTablet for tablet 15946ab2a3894ef7a4a719144a674304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:45:13 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:36992
I20260812 14:45:13.951233 14332 tablet_replica.cc:330] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: stopping tablet replica
I20260812 14:45:13.951331 14332 raft_consensus.cc:2229] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:45:13.951404 14332 raft_consensus.cc:2258] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:45:13.953249 14332 ts_tablet_manager.cc:1905] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:45:13.954676 14332 ts_tablet_manager.cc:1918] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:45:13.954805 14332 log.cc:1196] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/15946ab2a3894ef7a4a719144a674304
I20260812 14:45:13.955085 14332 ts_tablet_manager.cc:1939] T 15946ab2a3894ef7a4a719144a674304 P ed8e402316f042f090bbfb56846624e0: Deleting consensus metadata
I20260812 14:45:13.955495 14331 ts_tablet_manager.cc:1905] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:45:13.957093 14331 ts_tablet_manager.cc:1918] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:45:13.957209 14331 log.cc:1196] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/15946ab2a3894ef7a4a719144a674304
I20260812 14:45:13.957474 14331 ts_tablet_manager.cc:1939] T 15946ab2a3894ef7a4a719144a674304 P 1ea6f87083c4487485a473c99bf3d1ef: Deleting consensus metadata
I20260812 14:45:13.963052 13348 catalog_manager.cc:4841] TS ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929): tablet 15946ab2a3894ef7a4a719144a674304 (table test-table [id=dfbb5dd323274a008b8acddf4f069835]) successfully deleted
I20260812 14:45:13.963150 13348 catalog_manager.cc:4841] TS 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075): tablet 15946ab2a3894ef7a4a719144a674304 (table test-table [id=dfbb5dd323274a008b8acddf4f069835]) successfully deleted
W20260812 14:45:13.967391 13350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:33809 (error 108)
I20260812 14:45:13.967638 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 13767
I20260812 14:45:13.969251 14286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:13.967775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33809 (local address 127.0.0.1:59814)
0812 14:45:13.969164 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:13.969210 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1367}
W20260812 14:45:13.974192 13350 catalog_manager.cc:4584] TS 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 15946ab2a3894ef7a4a719144a674304: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33809: connect: Connection refused (error 111)
I20260812 14:45:13.975412 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 14015
I20260812 14:45:13.986840 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 13310
I20260812 14:45:33.997040 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33275
--webserver_interface=127.15.227.254
--webserver_port=41527
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:45:34.079663 14334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:34.079843 14334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:34.079864 14334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:34.081116 14334 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:45:34.081171 14334 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:45:34.081183 14334 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:45:34.081194 14334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:45:34.081207 14334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:34.081218 14334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:34.081230 14334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:45:34.081256 14334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:45:34.082803 14334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:34.083014 14334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:34.083220 14334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:34.091887 14353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:34.091902 14351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.091952 14334 server_base.cc:1034] running on GCE node
W20260812 14:45:34.091899 14348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.092350 14334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:34.092664 14334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:34.093807 14334 hybrid_clock.cc:648] HybridClock initialized: now 1786545934093793 us; error 35 us; skew 500 ppm
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:34.096200 14334 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:45:34.096630 14334 webserver.cc:460] Webserver started at http://127.15.227.254:41527/ using document root <none> and password file <none>
I20260812 14:45:34.096845 14334 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:34.096909 14334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:34.098186 14334 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:34.098783 14367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.098925 14334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:34.099012 14334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "2b11de877d554cc39cb809d4550b9fe6"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "3aef6dd2f109c24ceca7291bfb4c1a0e"
server_key_iv: "3b331564cc140d062a029d069d544dbf"
server_key_version: "encryptionkey@0"
I20260812 14:45:34.099362 14334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:34.108732 14334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:34.110425 14334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:34.110548 14334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:34.114409 14334 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:33275
I20260812 14:45:34.114439 14448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:33275 every 8 connection(s)
I20260812 14:45:34.114782 14334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:45:34.116153 14450 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:45:34.116739 14450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Bootstrap starting.
I20260812 14:45:34.122828 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 14334
I20260812 14:45:34.123148 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:33809
--local_ip_for_outbound_sockets=127.15.227.193
--tserver_master_addrs=127.15.227.254:33275
--webserver_port=44815
--webserver_interface=127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:45:34.126180 14450 log.cc:824] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:34.127902 14450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:45:34.128177 14450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Bootstrap complete.
I20260812 14:45:34.130384 14450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:34.130597 14450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b11de877d554cc39cb809d4550b9fe6, State: Initialized, Role: FOLLOWER
I20260812 14:45:34.130731 14450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:34.130789 14450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:45:34.130821 14450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:45:34.130868 14450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 14:45:34.131798 14450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:34.131882 14450 leader_election.cc:302] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b11de877d554cc39cb809d4550b9fe6; no voters:
I20260812 14:45:34.132019 14450 leader_election.cc:288] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 14:45:34.132064 14460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 14:45:34.132222 14460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 LEADER]: Becoming Leader. State: Replica: 2b11de877d554cc39cb809d4550b9fe6, State: Running, Role: LEADER
I20260812 14:45:34.132305 14450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:45:34.132357 14460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:34.132699 14461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2b11de877d554cc39cb809d4550b9fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } } }
I20260812 14:45:34.132789 14461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:45:34.132730 14460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b11de877d554cc39cb809d4550b9fe6. Latest consensus state: current_term: 2 leader_uuid: "2b11de877d554cc39cb809d4550b9fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } } }
I20260812 14:45:34.132949 14460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:45:34.133370 14474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:45:34.134181 14474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:45:34.134284 14474 catalog_manager.cc:1259] Loaded cluster ID: dcbed9ebcdca4e86a70f16750cd1fa8a
I20260812 14:45:34.134315 14474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:45:34.135438 14474 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:45:34.135625 14474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Loaded TSK: 0
I20260812 14:45:34.146106 14474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Generated new TSK 1
I20260812 14:45:34.146194 14474 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:45:34.205500 14455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:34.205686 14455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:34.205718 14455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:34.206856 14455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:34.206895 14455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:34.206929 14455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:45:34.208353 14455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:33809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=44815
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:34.208561 14455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:34.208761 14455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:34.213388 14507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:34.213438 14513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.213617 14455 server_base.cc:1034] running on GCE node
W20260812 14:45:34.213711 14515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.213924 14455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:34.214217 14455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:34.215397 14455 hybrid_clock.cc:648] HybridClock initialized: now 1786545934215385 us; error 28 us; skew 500 ppm
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:34.218334 14455 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:45:34.218875 14455 webserver.cc:460] Webserver started at http://127.15.227.193:44815/ using document root <none> and password file <none>
I20260812 14:45:34.219110 14455 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:34.219197 14455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:34.220832 14455 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:34.221542 14525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.221697 14455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:34.221773 14455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "73479b3759e745aebc58100a407dc5fb"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "86b4a4cfd5368ea843c8927095dcb9b2"
server_key_iv: "64782488e502e01a51dd953b35b52fd2"
server_key_version: "encryptionkey@0"
I20260812 14:45:34.222106 14455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:34.233592 14455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:34.235600 14455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:34.235710 14455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:34.235926 14455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:34.236332 14537 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:45:34.238201 14455 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:45:34.238248 14455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:45:34.238281 14455 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:45:34.238909 14455 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:45:34.238942 14455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260812 14:45:34.239029 14537 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Bootstrap starting.
I20260812 14:45:34.245675 14455 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:33809
I20260812 14:45:34.246037 14455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:45:34.246284 14731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:33809 every 8 connection(s)
I20260812 14:45:34.248664 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 14455
I20260812 14:45:34.248934 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:39075
--local_ip_for_outbound_sockets=127.15.227.194
--tserver_master_addrs=127.15.227.254:33275
--webserver_port=42991
--webserver_interface=127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:45:34.249989 14537 log.cc:824] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:34.252480 14537 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:45:34.253238 14537 tablet_bootstrap.cc:492] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Bootstrap complete.
I20260812 14:45:34.254458 14537 ts_tablet_manager.cc:1397] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20260812 14:45:34.255761 14537 raft_consensus.cc:348] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:34.256170 14537 raft_consensus.cc:729] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73479b3759e745aebc58100a407dc5fb, State: Initialized, Role: FOLLOWER
I20260812 14:45:34.256383 14537 consensus_queue.cc:260] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } } peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } }
I20260812 14:45:34.256752 14537 ts_tablet_manager.cc:1428] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:45:34.264993 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.252938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:60189)
0812 14:45:34.253246 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.253295 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.253886 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.254391 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.254395 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.254719 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:34.255003 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.255010 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.256639 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.256641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.257863 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.257867 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.258737 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.258739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.258878 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.259484 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:34.259493 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:34.261237 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:34.263271 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.263279 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.263281 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.263560 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.263826 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.263827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.263827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.263947 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.264379 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:34.264385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:34.264530 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.264639 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.264788 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":77,"threads_started":1}
I20260812 14:45:34.265008 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.253037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:60189 (local address 127.15.227.254:33275)
0812 14:45:34.254041 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.254050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.254081 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:45:34.254181 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.254185 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.254232 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.254346 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:34.255143 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.256514 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.257967 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.258617 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.258738 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.261342 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:34.261346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:34.261346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:34.261360 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:34.263199 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.263646 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.263649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.263649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.263758 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.264009 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.264010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.264011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.264305 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:34.264383 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.264707 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.264846 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":924,"thread_start_us":86,"threads_started":1}
I20260812 14:45:34.266094 14734 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33275
I20260812 14:45:34.266186 14734 heartbeater.cc:464] Registering TS with master...
I20260812 14:45:34.266345 14734 heartbeater.cc:511] Master 127.15.227.254:33275 requested a full tablet report, sending...
I20260812 14:45:34.267074 14392 ts_manager.cc:194] Registered new tserver with Master: 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809)
I20260812 14:45:34.268160 14392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:60189
I20260812 14:45:34.268945 14734 heartbeater.cc:503] Master 127.15.227.254:33275 was elected leader, sending a full tablet report...
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.193@KRBTEST.COM
I20260812 14:45:34.275331 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.267666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33809 (local address 127.0.0.1:60376)
0812 14:45:34.268081 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.268114 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.268191 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.268414 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.268420 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.268733 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:34.268786 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.268789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.269733 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.269735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.270658 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.270660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.271044 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.271045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.271115 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.271476 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:34.271481 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:34.272787 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:34.274299 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.274305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.274307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.274491 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.274677 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.274677 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.274678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.274741 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.274995 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:34.275001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:34.275051 (+ 50us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.275123 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.275237 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":76,"threads_started":1}
I20260812 14:45:34.275339 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.267895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60376 (local address 127.15.227.193:33809)
0812 14:45:34.268057 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.268062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.268179 (+ 117us) server_negotiation.cc:407] Connection header received
0812 14:45:34.268316 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.268319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.268334 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.268397 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:34.268873 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.269668 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.270726 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.270985 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.271036 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.272865 (+ 1829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:34.272872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:34.272872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:34.272879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:34.274219 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.274563 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.274566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.274567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.274620 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.274802 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.274803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.274803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.274930 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:34.275002 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.275167 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.275259 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
I20260812 14:45:34.275661 14597 tablet_service.cc:1514] Processing DeleteTablet for tablet 15946ab2a3894ef7a4a719144a674304 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:45:13 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:60376
I20260812 14:45:34.276130 14745 tablet_replica.cc:330] stopping tablet replica
I20260812 14:45:34.276230 14745 raft_consensus.cc:2229] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:45:34.276290 14745 raft_consensus.cc:2258] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:45:34.276579 14745 ts_tablet_manager.cc:1905] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:45:34.278652 14745 ts_tablet_manager.cc:1918] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:45:34.278726 14745 log.cc:1196] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal/wals/15946ab2a3894ef7a4a719144a674304
I20260812 14:45:34.278935 14745 ts_tablet_manager.cc:1939] T 15946ab2a3894ef7a4a719144a674304 P 73479b3759e745aebc58100a407dc5fb: Deleting consensus metadata
I20260812 14:45:34.279412 14371 catalog_manager.cc:4841] TS 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809): tablet 15946ab2a3894ef7a4a719144a674304 (table test-table [id=dfbb5dd323274a008b8acddf4f069835]) successfully deleted
W20260812 14:45:34.337749 14738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:34.337924 14738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:34.337945 14738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:34.339076 14738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:34.339118 14738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:34.339143 14738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:45:34.340515 14738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:39075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=42991
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:34.340710 14738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:34.340905 14738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:34.345517 14752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:34.345542 14755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.345654 14738 server_base.cc:1034] running on GCE node
W20260812 14:45:34.345530 14753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.345906 14738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:34.346166 14738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:34.347304 14738 hybrid_clock.cc:648] HybridClock initialized: now 1786545934347293 us; error 29 us; skew 500 ppm
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:34.349632 14738 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:45:34.350005 14738 webserver.cc:460] Webserver started at http://127.15.227.194:42991/ using document root <none> and password file <none>
I20260812 14:45:34.350212 14738 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:34.350267 14738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:34.351444 14738 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:45:34.351955 14764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.352106 14738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.352207 14738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "1ea6f87083c4487485a473c99bf3d1ef"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "6c2ba386d0d06d1c3436680c6781f360"
server_key_iv: "5a6dd94aafac4626e77035ce5814dd14"
server_key_version: "encryptionkey@0"
I20260812 14:45:34.352480 14738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:34.371850 14738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:34.373713 14738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:34.373816 14738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:34.373979 14738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:34.374269 14738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:45:34.374303 14738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.374322 14738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:45:34.374338 14738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.379629 14738 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:39075
I20260812 14:45:34.379690 14970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:39075 every 8 connection(s)
I20260812 14:45:34.379962 14738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:45:34.387391 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 14738
I20260812 14:45:34.387632 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:37929
--local_ip_for_outbound_sockets=127.15.227.195
--tserver_master_addrs=127.15.227.254:33275
--webserver_port=35019
--webserver_interface=127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:45:34.388149 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.380559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:39607 (local address 127.15.227.254:33275)
0812 14:45:34.380599 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.380601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.380808 (+ 207us) server_negotiation.cc:407] Connection header received
0812 14:45:34.381268 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.381270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.381290 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.381341 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:34.382250 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.382748 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.383696 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.383905 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.383962 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.385763 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:34.385765 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:34.385765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:34.385773 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:34.386878 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.387227 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.387228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.387229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.387287 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.387537 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.387539 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.387539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.387648 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:34.387713 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.388012 (+ 299us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.388087 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:45:34.388325 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.380547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:39607)
0812 14:45:34.380695 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.380728 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.381167 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.381406 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.381410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.381934 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:34.382161 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.382166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.382831 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.382832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.383609 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.383612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.383977 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.383978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.384102 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.384456 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:34.384463 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:34.385667 (+ 1204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:34.386951 (+ 1284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.386958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.386960 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.387160 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.387341 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.387342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.387343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.387451 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.387723 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:34.387732 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:34.387877 (+ 145us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.387983 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.388126 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260812 14:45:34.388779 14976 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33275
I20260812 14:45:34.388859 14976 heartbeater.cc:464] Registering TS with master...
I20260812 14:45:34.389019 14976 heartbeater.cc:511] Master 127.15.227.254:33275 requested a full tablet report, sending...
I20260812 14:45:34.389387 14392 ts_manager.cc:194] Registered new tserver with Master: 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
I20260812 14:45:34.389740 14392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:39607
W20260812 14:45:34.468848 14982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:34.469022 14982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:34.469043 14982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:34.470170 14982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:34.470208 14982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:34.470237 14982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:45:34.471632 14982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:37929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=35019
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:34.471835 14982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:34.472029 14982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:34.476653 14996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:34.476733 14992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.476814 14982 server_base.cc:1034] running on GCE node
W20260812 14:45:34.476761 14989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:34.477058 14982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:34.477277 14982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:34.478411 14982 hybrid_clock.cc:648] HybridClock initialized: now 1786545934478396 us; error 29 us; skew 500 ppm
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:34.480645 14982 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:45:34.481030 14982 webserver.cc:460] Webserver started at http://127.15.227.195:35019/ using document root <none> and password file <none>
I20260812 14:45:34.481212 14982 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:34.481277 14982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:34.482558 14982 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:34.483155 15012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.483304 14982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:34.483381 14982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "ed8e402316f042f090bbfb56846624e0"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "fc07e66e8e26150ce6e54ffd516364a6"
server_key_iv: "3ce8512abb7c505a694699a6c46b1ce4"
server_key_version: "encryptionkey@0"
I20260812 14:45:34.483651 14982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:34.500808 14982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:34.502549 14982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:34.502681 14982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:34.502866 14982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:34.503130 14982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:45:34.503162 14982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.503189 14982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:45:34.503206 14982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.508505 14982 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:37929
I20260812 14:45:34.508543 15212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:37929 every 8 connection(s)
I20260812 14:45:34.508845 14982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:45:34.512872 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 14982
Aug 12 14:45:34 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545934, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:45:34.517491 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.509482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:44827 (local address 127.15.227.254:33275)
0812 14:45:34.509524 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.509526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.509705 (+ 179us) server_negotiation.cc:407] Connection header received
0812 14:45:34.510145 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.510146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.510158 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.510206 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:34.511047 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.511529 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.512417 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.512616 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.512669 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.514827 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:34.514830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:34.514830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:34.514838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:34.516127 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.516599 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.516603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.516604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.516661 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.516911 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.516912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.516912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.517002 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:34.517048 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.517342 (+ 294us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.517426 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:45:34.517664 15215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.509468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:44827)
0812 14:45:34.509593 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.509625 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.510065 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.510255 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.510258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.510774 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:34.510946 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.510950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.511599 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.511600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.512338 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.512340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.512681 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.512682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.512825 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.513360 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:34.513370 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:34.514738 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:34.516215 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.516230 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.516232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.516514 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.516724 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:34.516724 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:34.516725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:34.516854 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:34.517048 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:34.517054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:34.517205 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.517305 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.517454 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":67,"threads_started":1}
I20260812 14:45:34.518224 15213 heartbeater.cc:347] Connected to a master server at 127.15.227.254:33275
I20260812 14:45:34.518321 15213 heartbeater.cc:464] Registering TS with master...
I20260812 14:45:34.518507 15213 heartbeater.cc:511] Master 127.15.227.254:33275 requested a full tablet report, sending...
I20260812 14:45:34.518905 14392 ts_manager.cc:194] Registered new tserver with Master: ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929)
I20260812 14:45:34.519261 14392 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:44827
I20260812 14:45:34.519673 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.513265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58458 (local address 127.15.227.254:33275)
0812 14:45:34.513402 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.513404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.513536 (+ 132us) server_negotiation.cc:407] Connection header received
0812 14:45:34.513586 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.513587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.513599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.513657 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:34.514250 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.514769 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.515512 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.515715 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.515772 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.516991 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:34.516993 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:34.516993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:34.517000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:34.518533 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.518897 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.518898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.518898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.518990 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.519157 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.519158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.519159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.519273 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:34.519430 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.519506 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.519586 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260812 14:45:34.520538 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:45:34.522949 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.520847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58474 (local address 127.15.227.254:33275)
0812 14:45:34.520881 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.520884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.520989 (+ 105us) server_negotiation.cc:407] Connection header received
0812 14:45:34.521041 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.521041 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.521067 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.521124 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:45:34.521195 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.521629 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.522377 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.522558 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.522603 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.522722 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:45:34.522766 (+ 44us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 14:45:34.522902 (+ 136us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58474: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:45:34.523001 15216 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58474: token expired
W20260812 14:45:34.523162 14276 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.227.254:33275)
I20260812 14:45:34.527832 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.523358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.15.227.254:33275)
0812 14:45:34.523393 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.523394 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.523402 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:45:34.523419 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.523420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.523427 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.523468 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:34.523840 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.524290 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.524870 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.525115 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.525158 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.526086 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:34.526088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:34.526088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:34.526094 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:34.527089 (+ 995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.527371 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.527372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.527372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.527412 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:34.527525 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:34.527525 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:34.527525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:34.527602 (+ 77us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:34.527660 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.527721 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.527781 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:45:34.528839 14392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:45:34.530454 14392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:45:34.532397 14597 tablet_service.cc:1467] Processing CreateTablet for tablet fda1420c23e64576bb798b4809e6cf12 (DEFAULT_TABLE table=test-table [id=aa17077676f84aea92eed1fdbedd3dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:45:34.532595 14597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fda1420c23e64576bb798b4809e6cf12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:45:34.534159 14537 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Bootstrap starting.
I20260812 14:45:34.534497 14537 tablet_bootstrap.cc:654] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Neither blocks nor log segments found. Creating new log.
I20260812 14:45:34.535331 14537 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: No bootstrap required, opened a new log
I20260812 14:45:34.535389 14537 ts_tablet_manager.cc:1397] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:34.535537 14537 raft_consensus.cc:348] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.535586 14537 raft_consensus.cc:374] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:45:34.535614 14537 raft_consensus.cc:729] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73479b3759e745aebc58100a407dc5fb, State: Initialized, Role: FOLLOWER
I20260812 14:45:34.535650 14537 consensus_queue.cc:260] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.535719 14537 ts_tablet_manager.cc:1428] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:34.537384 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.532109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:37929 (local address 127.0.0.1:35878)
0812 14:45:34.532338 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.532347 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.532387 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.534285 (+ 1898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.534286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.534288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:34.534326 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.534327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.535451 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.535452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.536425 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.536427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.537126 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.537127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.537197 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.537217 (+ 20us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.537274 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.537318 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260812 14:45:34.537396 15223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.532184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.15.227.195:37929)
0812 14:45:34.534131 (+ 1947us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.534135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.534144 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:45:34.534191 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.534194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.534208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.534292 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:34.534366 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.535359 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.536494 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.537054 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.537116 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.537177 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.537292 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.537326 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1912,"thread_start_us":90,"threads_started":1}
I20260812 14:45:34.537719 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.532204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:39075 (local address 127.0.0.1:56850)
0812 14:45:34.532514 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.532523 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.532572 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.532783 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.532784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.532786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:34.532822 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.532824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.535090 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.535091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.536694 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.536697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.537482 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.537483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.537549 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.537553 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.537631 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.537668 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":98,"threads_started":1}
I20260812 14:45:34.537890 15227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.532327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56850 (local address 127.15.227.194:39075)
0812 14:45:34.532504 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.532509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.532566 (+ 57us) server_negotiation.cc:407] Connection header received
0812 14:45:34.532651 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.532654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.532671 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.532765 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:34.532893 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.533980 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.536775 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.537236 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.537308 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.537367 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.537668 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.537719 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260812 14:45:34.538087 15095 tablet_service.cc:1467] Processing CreateTablet for tablet fda1420c23e64576bb798b4809e6cf12 (DEFAULT_TABLE table=test-table [id=aa17077676f84aea92eed1fdbedd3dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:45:34.538420 15095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fda1420c23e64576bb798b4809e6cf12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:45:34.538583 14853 tablet_service.cc:1467] Processing CreateTablet for tablet fda1420c23e64576bb798b4809e6cf12 (DEFAULT_TABLE table=test-table [id=aa17077676f84aea92eed1fdbedd3dae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:45:34.538918 14853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fda1420c23e64576bb798b4809e6cf12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:45:34.541796 15230 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Bootstrap starting.
I20260812 14:45:34.542210 15232 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Bootstrap starting.
I20260812 14:45:34.542469 15230 tablet_bootstrap.cc:654] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Neither blocks nor log segments found. Creating new log.
I20260812 14:45:34.542806 15232 tablet_bootstrap.cc:654] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Neither blocks nor log segments found. Creating new log.
I20260812 14:45:34.542963 15230 log.cc:824] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:34.543329 15232 log.cc:824] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:34.543730 15230 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: No bootstrap required, opened a new log
I20260812 14:45:34.543804 15230 ts_tablet_manager.cc:1397] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:45:34.544070 15232 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: No bootstrap required, opened a new log
I20260812 14:45:34.544122 15232 ts_tablet_manager.cc:1397] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:45:34.545258 15230 raft_consensus.cc:348] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.545390 15230 raft_consensus.cc:374] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:45:34.545423 15230 raft_consensus.cc:729] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed8e402316f042f090bbfb56846624e0, State: Initialized, Role: FOLLOWER
I20260812 14:45:34.545538 15230 consensus_queue.cc:260] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.545585 15232 raft_consensus.cc:348] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.545766 15232 raft_consensus.cc:374] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:45:34.545805 15230 ts_tablet_manager.cc:1428] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:45:34.545818 15232 raft_consensus.cc:729] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ea6f87083c4487485a473c99bf3d1ef, State: Initialized, Role: FOLLOWER
I20260812 14:45:34.545831 15213 heartbeater.cc:503] Master 127.15.227.254:33275 was elected leader, sending a full tablet report...
I20260812 14:45:34.545948 15232 consensus_queue.cc:260] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.546276 14976 heartbeater.cc:503] Master 127.15.227.254:33275 was elected leader, sending a full tablet report...
I20260812 14:45:34.546476 15232 ts_tablet_manager.cc:1428] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260812 14:45:34.630486 14980 tablet.cc:2369] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:45:34.668498 15238 raft_consensus.cc:482] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:34.668620 15238 raft_consensus.cc:504] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.668951 15238 leader_election.cc:288] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929), 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809)
I20260812 14:45:34.671739 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.669130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:37929 (local address 127.15.227.194:44169)
0812 14:45:34.669165 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.669177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.669224 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.669402 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.669403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.669405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:34.669477 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.669479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.670169 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.670170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.670907 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.670909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.671483 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.671485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.671566 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.671571 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.671629 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.671667 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260812 14:45:34.671748 15223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.669228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:44169 (local address 127.15.227.195:37929)
0812 14:45:34.669277 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.669279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.669287 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:45:34.669315 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.669316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.669325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.669390 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:34.669543 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.670093 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.670991 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.671391 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.671446 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.671455 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.671639 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.671679 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260812 14:45:34.671819 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.669172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:54989 (local address 127.15.227.193:33809)
0812 14:45:34.669221 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.669233 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.669396 (+ 163us) server_negotiation.cc:407] Connection header received
0812 14:45:34.669434 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.669435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.669466 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.669536 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:45:34.669674 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.670189 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.671079 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.671505 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.671581 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.671677 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.671713 (+ 36us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.671758 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:45:34.671844 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.669131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33809 (local address 127.15.227.194:54989)
0812 14:45:34.669323 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.669333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:34.669390 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:34.669543 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:34.669543 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:34.669545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:45:34.669609 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.669611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.670259 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.670260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.670979 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:34.670981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.671588 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.671588 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:34.671656 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.671660 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:45:34.671696 (+ 36us) client_negotiation.cc:240] Negotiation successful
0812 14:45:34.671740 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":40,"thread_start_us":94,"threads_started":1}
I20260812 14:45:34.672254 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fda1420c23e64576bb798b4809e6cf12" candidate_uuid: "1ea6f87083c4487485a473c99bf3d1ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed8e402316f042f090bbfb56846624e0" is_pre_election: true
I20260812 14:45:34.672250 14649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fda1420c23e64576bb798b4809e6cf12" candidate_uuid: "1ea6f87083c4487485a473c99bf3d1ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73479b3759e745aebc58100a407dc5fb" is_pre_election: true
I20260812 14:45:34.672384 14649 raft_consensus.cc:2454] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ea6f87083c4487485a473c99bf3d1ef in term 0.
I20260812 14:45:34.672400 15121 raft_consensus.cc:2454] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ea6f87083c4487485a473c99bf3d1ef in term 0.
I20260812 14:45:34.672591 14769 leader_election.cc:302] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ea6f87083c4487485a473c99bf3d1ef, 73479b3759e745aebc58100a407dc5fb; no voters:
I20260812 14:45:34.672760 15238 raft_consensus.cc:2781] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:45:34.672816 15238 raft_consensus.cc:482] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:45:34.672832 15238 raft_consensus.cc:3037] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:45:34.673593 15238 raft_consensus.cc:504] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.673710 15238 leader_election.cc:288] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [CANDIDATE]: Term 1 election: Requested vote from peers ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929), 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809)
I20260812 14:45:34.673853 14649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fda1420c23e64576bb798b4809e6cf12" candidate_uuid: "1ea6f87083c4487485a473c99bf3d1ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73479b3759e745aebc58100a407dc5fb"
I20260812 14:45:34.673862 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fda1420c23e64576bb798b4809e6cf12" candidate_uuid: "1ea6f87083c4487485a473c99bf3d1ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed8e402316f042f090bbfb56846624e0"
I20260812 14:45:34.673928 14649 raft_consensus.cc:3037] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:45:34.673934 15121 raft_consensus.cc:3037] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:45:34.674607 14649 raft_consensus.cc:2454] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ea6f87083c4487485a473c99bf3d1ef in term 1.
I20260812 14:45:34.674813 14769 leader_election.cc:302] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ea6f87083c4487485a473c99bf3d1ef, 73479b3759e745aebc58100a407dc5fb; no voters:
I20260812 14:45:34.674817 15121 raft_consensus.cc:2454] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ea6f87083c4487485a473c99bf3d1ef in term 1.
I20260812 14:45:34.674902 15238 raft_consensus.cc:2781] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:45:34.675042 15238 raft_consensus.cc:686] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 LEADER]: Becoming Leader. State: Replica: 1ea6f87083c4487485a473c99bf3d1ef, State: Running, Role: LEADER
I20260812 14:45:34.675124 15238 consensus_queue.cc:237] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:34.675868 14390 catalog_manager.cc:5495] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194). New cstate: current_term: 1 leader_uuid: "1ea6f87083c4487485a473c99bf3d1ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } health_report { overall_health: UNKNOWN } } }
I20260812 14:45:34.682147 15227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.680328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.15.227.194:39075)
0812 14:45:34.680365 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:34.680367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:34.680376 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:45:34.680403 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:34.680403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:34.680417 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:34.680468 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:45:34.680551 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.680960 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.681608 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:34.681780 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:34.681818 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:34.681908 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:45:34.681986 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:45:34.682063 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:34.682077 (+ 14us) server_negotiation.cc:299] Negotiation successful
0812 14:45:34.682091 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:45:34.683561 15121 raft_consensus.cc:1261] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 1 FOLLOWER]: Refusing update from remote peer 1ea6f87083c4487485a473c99bf3d1ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:45:34.683573 14649 raft_consensus.cc:1261] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Refusing update from remote peer 1ea6f87083c4487485a473c99bf3d1ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:45:34.683847 15238 consensus_queue.cc:1035] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:45:34.683962 15241 consensus_queue.cc:1035] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:45:34.685873 15249 mvcc.cc:205] Tried to move back new op lower bound from 7317692148461903872 to 7317692148429832192. Current Snapshot: MvccSnapshot[applied={T|T < 7317692148461903872}]
I20260812 14:45:34.686456 15251 mvcc.cc:205] Tried to move back new op lower bound from 7317692148461903872 to 7317692148429832192. Current Snapshot: MvccSnapshot[applied={T|T < 7317692148461903872}]
I20260812 14:45:34.687866 15252 mvcc.cc:205] Tried to move back new op lower bound from 7317692148461903872 to 7317692148429832192. Current Snapshot: MvccSnapshot[applied={T|T < 7317692148461903872 or (T in {7317692148461903872})}]
I20260812 14:45:34.688164 14390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:45:34.688248 14390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:45:34.688859 14390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Sending DeleteTablet for 3 replicas of tablet fda1420c23e64576bb798b4809e6cf12
I20260812 14:45:34.689134 14597 tablet_service.cc:1514] Processing DeleteTablet for tablet fda1420c23e64576bb798b4809e6cf12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:45:34 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:60376
I20260812 14:45:34.689232 14745 tablet_replica.cc:330] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: stopping tablet replica
I20260812 14:45:34.689236 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 14455
I20260812 14:45:34.689232 15095 tablet_service.cc:1514] Processing DeleteTablet for tablet fda1420c23e64576bb798b4809e6cf12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:45:34 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:35878
I20260812 14:45:34.689280 14745 raft_consensus.cc:2229] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:45:34.689317 14745 raft_consensus.cc:2258] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:45:34.710433 15264 tablet_replica.cc:330] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: stopping tablet replica
I20260812 14:45:34.710640 15264 raft_consensus.cc:2229] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:45:34.710841 15264 raft_consensus.cc:2258] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:45:34.711565 14853 tablet_service.cc:1514] Processing DeleteTablet for tablet fda1420c23e64576bb798b4809e6cf12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:45:34 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:56850
I20260812 14:45:34.713508 15268 tablet_replica.cc:330] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: stopping tablet replica
I20260812 14:45:34.713682 15268 raft_consensus.cc:2229] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:45:34.713858 15268 raft_consensus.cc:2258] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:45:34.716790 15268 ts_tablet_manager.cc:1905] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:45:34.718873 15268 ts_tablet_manager.cc:1918] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:45:34.719028 15268 log.cc:1196] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/fda1420c23e64576bb798b4809e6cf12
I20260812 14:45:34.719489 15268 ts_tablet_manager.cc:1939] T fda1420c23e64576bb798b4809e6cf12 P 1ea6f87083c4487485a473c99bf3d1ef: Deleting consensus metadata
I20260812 14:45:34.720068 14369 catalog_manager.cc:4841] TS 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075): tablet fda1420c23e64576bb798b4809e6cf12 (table test-table [id=aa17077676f84aea92eed1fdbedd3dae]) successfully deleted
I20260812 14:45:34.721340 15264 ts_tablet_manager.cc:1905] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:45:34.723259 15264 ts_tablet_manager.cc:1918] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:45:34.723325 15264 log.cc:1196] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/fda1420c23e64576bb798b4809e6cf12
I20260812 14:45:34.723671 15264 ts_tablet_manager.cc:1939] T fda1420c23e64576bb798b4809e6cf12 P ed8e402316f042f090bbfb56846624e0: Deleting consensus metadata
I20260812 14:45:34.724181 14369 catalog_manager.cc:4841] TS ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929): tablet fda1420c23e64576bb798b4809e6cf12 (table test-table [id=aa17077676f84aea92eed1fdbedd3dae]) successfully deleted
W20260812 14:45:34.734262 14371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:33809 (error 108)
I20260812 14:45:34.734488 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 14738
I20260812 14:45:34.736287 15224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:34.736166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33809 (local address 127.0.0.1:60388)
0812 14:45:34.736211 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:34.736253 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:45:34.736466 14371 catalog_manager.cc:4584] TS 73479b3759e745aebc58100a407dc5fb (127.15.227.193:33809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fda1420c23e64576bb798b4809e6cf12: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33809: connect: Connection refused (error 111)
I20260812 14:45:34.742709 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 14982
I20260812 14:45:34.750834 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 14334
I20260812 14:45:54.763206 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33275
--webserver_interface=127.15.227.254
--webserver_port=41527
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:45:54.845909 15270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:54.846113 15270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:54.846134 15270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:54.847258 15270 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 14:45:54.847302 15270 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 14:45:54.847314 15270 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 14:45:54.847324 15270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:45:54.847337 15270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:54.847348 15270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:54.847359 15270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:45:54.847375 15270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:45:54.848860 15270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=41527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:54.849079 15270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:54.849296 15270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:54.854182 15278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:54.854197 15270 server_base.cc:1034] running on GCE node
W20260812 14:45:54.854270 15282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:54.854300 15288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:54.854640 15270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:54.854897 15270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:54.856052 15270 hybrid_clock.cc:648] HybridClock initialized: now 1786545954856030 us; error 40 us; skew 500 ppm
Aug 12 14:45:54 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:54.858426 15270 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:45:54.858853 15270 webserver.cc:460] Webserver started at http://127.15.227.254:41527/ using document root <none> and password file <none>
I20260812 14:45:54.859066 15270 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:54.859114 15270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:54.860369 15270 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:54.860955 15299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:54.861110 15270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:54.861205 15270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "2b11de877d554cc39cb809d4550b9fe6"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "3aef6dd2f109c24ceca7291bfb4c1a0e"
server_key_iv: "3b331564cc140d062a029d069d544dbf"
server_key_version: "encryptionkey@0"
I20260812 14:45:54.861480 15270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:54.871666 15270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:54.873227 15270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:54.873349 15270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:54.877763 15270 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:33275
I20260812 14:45:54.877818 15398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:33275 every 8 connection(s)
I20260812 14:45:54.878196 15270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:45:54.879254 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 15270
I20260812 14:45:54.879519 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:33809
--local_ip_for_outbound_sockets=127.15.227.193
--tserver_master_addrs=127.15.227.254:33275
--webserver_port=44815
--webserver_interface=127.15.227.193
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:45:54.879700 15399 sys_catalog.cc:263] Verifying existing consensus state
I20260812 14:45:54.880307 15399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Bootstrap starting.
I20260812 14:45:54.889333 15399 log.cc:824] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:54.891414 15399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:45:54.891674 15399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Bootstrap complete.
I20260812 14:45:54.893790 15399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:54.893998 15399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b11de877d554cc39cb809d4550b9fe6, State: Initialized, Role: FOLLOWER
I20260812 14:45:54.894181 15399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:54.894254 15399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:45:54.894285 15399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:45:54.894331 15399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 14:45:54.895248 15399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:54.895331 15399 leader_election.cc:302] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b11de877d554cc39cb809d4550b9fe6; no voters:
I20260812 14:45:54.895481 15399 leader_election.cc:288] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 14:45:54.895529 15405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 14:45:54.895648 15405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [term 3 LEADER]: Becoming Leader. State: Replica: 2b11de877d554cc39cb809d4550b9fe6, State: Running, Role: LEADER
I20260812 14:45:54.895735 15399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:45:54.895804 15405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } }
I20260812 14:45:54.896090 15407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2b11de877d554cc39cb809d4550b9fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } } }
I20260812 14:45:54.896130 15405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b11de877d554cc39cb809d4550b9fe6. Latest consensus state: current_term: 3 leader_uuid: "2b11de877d554cc39cb809d4550b9fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b11de877d554cc39cb809d4550b9fe6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33275 } } }
I20260812 14:45:54.896247 15407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:45:54.896275 15405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:45:54.896518 15419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:45:54.897456 15419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:45:54.897620 15419 catalog_manager.cc:1259] Loaded cluster ID: dcbed9ebcdca4e86a70f16750cd1fa8a
I20260812 14:45:54.897805 15419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:45:54.898655 15419 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:45:54.898849 15419 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Loaded TSK: 0
I20260812 14:45:54.898888 15419 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2b11de877d554cc39cb809d4550b9fe6: Loaded TSK: 1
F20260812 14:45:54.899094 15419 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd2032669ad google::LogMessage::Fail() at ??:0
@ 0x7fd20326988a google::LogMessage::SendToLog() at ??:0
@ 0x7fd2032664de google::LogMessage::Flush() at ??:0
@ 0x7fd203267a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd2059d3104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fd2059d0d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fd20b1c01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fd20b1c048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fd20b1c0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fd20b186c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fd20b190ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd2051340d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd205129954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd205f8d6db start_thread at ??:0
@ 0x7fd201d6461f clone at ??:0
W20260812 14:45:54.974376 15401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:54.974552 15401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:54.974584 15401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:54.975688 15401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:54.975728 15401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:54.975762 15401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:45:54.977181 15401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:33809
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=44815
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:54.977394 15401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:54.977586 15401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:54.982301 15473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:54.982364 15401 server_base.cc:1034] running on GCE node
W20260812 14:45:54.982502 15474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:54.982577 15478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:54.982798 15401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:54.983021 15401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:54.984170 15401 hybrid_clock.cc:648] HybridClock initialized: now 1786545954984157 us; error 34 us; skew 500 ppm
Aug 12 14:45:54 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:54.986487 15401 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:45:54.986897 15401 webserver.cc:460] Webserver started at http://127.15.227.193:44815/ using document root <none> and password file <none>
I20260812 14:45:54.987085 15401 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:54.987133 15401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:54.988394 15401 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:54.989041 15499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:54.989214 15401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:45:54.989293 15401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "73479b3759e745aebc58100a407dc5fb"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "86b4a4cfd5368ea843c8927095dcb9b2"
server_key_iv: "64782488e502e01a51dd953b35b52fd2"
server_key_version: "encryptionkey@0"
I20260812 14:45:54.989583 15401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:55.000880 15401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:55.002805 15401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:55.002913 15401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:55.003118 15401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:55.003530 15516 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260812 14:45:55.005352 15401 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 14:45:55.005393 15401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260812 14:45:55.005422 15401 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260812 14:45:55.005995 15401 ts_tablet_manager.cc:610] Registered 1 tablets
I20260812 14:45:55.006050 15401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:55.006145 15516 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Bootstrap starting.
I20260812 14:45:55.011966 15401 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:33809
I20260812 14:45:55.012377 15401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:45:55.013142 15702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:33809 every 8 connection(s)
I20260812 14:45:55.014855 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 15401
I20260812 14:45:55.015112 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:39075
--local_ip_for_outbound_sockets=127.15.227.194
--tserver_master_addrs=127.15.227.254:33275
--webserver_port=42991
--webserver_interface=127.15.227.194
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:45:55.017308 15516 log.cc:824] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Log is configured to *not* fsync() on all Append() calls
I20260812 14:45:55.019882 15516 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 14:45:55.020198 15516 tablet_bootstrap.cc:492] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Bootstrap complete.
I20260812 14:45:55.021369 15516 ts_tablet_manager.cc:1397] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260812 14:45:55.022745 15516 raft_consensus.cc:348] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:55.023056 15516 raft_consensus.cc:729] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73479b3759e745aebc58100a407dc5fb, State: Initialized, Role: FOLLOWER
I20260812 14:45:55.023234 15516 consensus_queue.cc:260] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:55.023499 15516 ts_tablet_manager.cc:1428] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb: Time spent starting tablet: real 0.002s user 0.004s sys 0.002s
W20260812 14:45:55.095556 15705 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:45:55.013608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:41229)
0812 14:45:55.014107 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.014136 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:55.014523 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:55.095401 (+ 80878us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":80,"threads_started":1}
W20260812 14:45:55.095806 15703 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:33275 (0 consecutive failures): Network error: Failed to ping master at 127.15.227.254:33275: Client connection negotiation failed: client connection to 127.15.227.254:33275: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:45:55.096139 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.096034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:46913)
0812 14:45:55.096080 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.096117 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 14:45:55.096473 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.096390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:60111)
0812 14:45:55.096429 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.096452 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 14:45:55.096589 15703 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:45:55.099385 15706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:55.099553 15706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:55.099575 15706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:55.100641 15706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:55.100673 15706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:55.100696 15706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:45:55.102098 15706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:39075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=42991
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:55.102283 15706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:55.102445 15706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:55.106997 15715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:55.107008 15716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:55.107013 15718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:55.107257 15706 server_base.cc:1034] running on GCE node
I20260812 14:45:55.107426 15706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:55.107622 15706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:55.108747 15706 hybrid_clock.cc:648] HybridClock initialized: now 1786545955108743 us; error 29 us; skew 500 ppm
Aug 12 14:45:55 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545955, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:55.111081 15706 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:45:55.111442 15706 webserver.cc:460] Webserver started at http://127.15.227.194:42991/ using document root <none> and password file <none>
I20260812 14:45:55.111616 15706 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:55.111656 15706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:55.112813 15706 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:45:55.113340 15730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:55.113507 15706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:55.113585 15706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "1ea6f87083c4487485a473c99bf3d1ef"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "6c2ba386d0d06d1c3436680c6781f360"
server_key_iv: "5a6dd94aafac4626e77035ce5814dd14"
server_key_version: "encryptionkey@0"
I20260812 14:45:55.113857 15706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:55.132838 15706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:55.134435 15706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:55.134539 15706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:55.134699 15706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:55.134943 15706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:45:55.134972 15706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:55.134990 15706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:45:55.135005 15706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:55.140576 15706 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:39075
I20260812 14:45:55.140653 15942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:39075 every 8 connection(s)
I20260812 14:45:55.140916 15706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:45:55.141177 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 15706
I20260812 14:45:55.141494 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:37929
--local_ip_for_outbound_sockets=127.15.227.195
--tserver_master_addrs=127.15.227.254:33275
--webserver_port=35019
--webserver_interface=127.15.227.195
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:45:55.142294 15947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.141546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:45251)
0812 14:45:55.142098 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.142155 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":83,"threads_started":1}
W20260812 14:45:55.142640 15944 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:33275 (0 consecutive failures): Network error: Failed to ping master at 127.15.227.254:33275: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
I20260812 14:45:55.142882 15947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.142799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:52229)
0812 14:45:55.142838 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.142864 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 14:45:55.143128 15947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.143042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:59639)
0812 14:45:55.143082 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.143106 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:45:55.143226 15944 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 14:45:55.221099 15950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:45:55.221277 15950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:45:55.221298 15950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:45:55.222421 15950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:45:55.222460 15950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:45:55.222486 15950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:45:55.223871 15950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:37929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=35019
--tserver_master_addrs=127.15.227.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:45:55.224092 15950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:45:55.224316 15950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:45:55.228983 15962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:55.229041 15950 server_base.cc:1034] running on GCE node
W20260812 14:45:55.229008 15967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:45:55.229008 15959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:45:55.229398 15950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:45:55.229591 15950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:45:55.230716 15950 hybrid_clock.cc:648] HybridClock initialized: now 1786545955230705 us; error 29 us; skew 500 ppm
Aug 12 14:45:55 dist-test-slave-q9h9 krb5kdc[13280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545955, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:45:55.232895 15950 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:45:55.233275 15950 webserver.cc:460] Webserver started at http://127.15.227.195:35019/ using document root <none> and password file <none>
I20260812 14:45:55.233456 15950 fs_manager.cc:359] Metadata directory not provided
I20260812 14:45:55.233501 15950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:45:55.234705 15950 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260812 14:45:55.235249 15977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:55.235392 15950 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:45:55.235471 15950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "ed8e402316f042f090bbfb56846624e0"
format_stamp: "Formatted at 2026-08-12 14:45:13 on dist-test-slave-q9h9"
server_key: "fc07e66e8e26150ce6e54ffd516364a6"
server_key_iv: "3ce8512abb7c505a694699a6c46b1ce4"
server_key_version: "encryptionkey@0"
I20260812 14:45:55.235760 15950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:45:55.256924 15950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:45:55.258610 15950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:45:55.258755 15950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:45:55.258970 15950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:45:55.259261 15950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:45:55.259297 15950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:55.259330 15950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:45:55.259356 15950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:45:55.265558 15950 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:37929
I20260812 14:45:55.265615 16208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:37929 every 8 connection(s)
I20260812 14:45:55.265924 15950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:45:55.266695 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 15950
I20260812 14:45:55.266956 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.266529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:43033)
0812 14:45:55.266689 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.266777 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
W20260812 14:45:55.267367 16209 heartbeater.cc:644] Failed to heartbeat to 127.15.227.254:33275 (0 consecutive failures): Network error: Failed to ping master at 127.15.227.254:33275: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
I20260812 14:45:55.267619 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.267531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:37251)
0812 14:45:55.267573 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.267598 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:45:55.267863 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:55.267788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:32881)
0812 14:45:55.267822 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:55.267844 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 14:45:55.267952 16209 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 14:45:56.097151 16216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:56.096920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:59009)
0812 14:45:56.097036 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:56.097103 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260812 14:45:56.143677 16217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:56.143492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:33193)
0812 14:45:56.143590 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:56.143621 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260812 14:45:56.268438 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:56.268254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:60823)
0812 14:45:56.268361 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:56.268393 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260812 14:45:56.621776 16219 raft_consensus.cc:482] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:56.621912 16219 raft_consensus.cc:504] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:56.622267 16219 leader_election.cc:288] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929), 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
Aug 12 14:45:56 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.194@KRBTEST.COM
Aug 12 14:45:56 dist-test-slave-q9h9 krb5kdc[13280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545954, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.195@KRBTEST.COM
I20260812 14:45:56.632030 16227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:56.622545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:39075 (local address 127.15.227.193:33961)
0812 14:45:56.622620 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:56.622628 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:56.622663 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:56.623483 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:56.623495 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:56.623850 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:56.624052 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:56.624058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.625137 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.625138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:56.625930 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:56.625933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.626532 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.626533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:56.626642 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:56.627187 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:56.627196 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:56.628557 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:56.630535 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:56.630541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:56.630543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:56.630761 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:56.630984 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:56.630984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:56.630985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:56.631085 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:56.631514 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:56.631519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:56.631659 (+ 140us) client_negotiation.cc:769] Sending connection context
0812 14:45:56.631765 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 14:45:56.631920 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":19,"thread_start_us":39,"threads_started":1}
I20260812 14:45:56.632055 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:56.622498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:60165 (local address 127.15.227.195:37929)
0812 14:45:56.622698 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:56.622711 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:56.622750 (+ 39us) server_negotiation.cc:407] Connection header received
0812 14:45:56.623223 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:56.623226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:56.623281 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:56.623397 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:56.624172 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.625028 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.626009 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.626440 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.626545 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:56.629026 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:56.629030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:56.629030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:56.629040 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:56.630719 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:56.631008 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:56.631010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:56.631011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:56.631089 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:56.631252 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:56.631253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:56.631253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:56.631509 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:56.631579 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:56.631814 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 14:45:56.631945 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":60,"threads_started":1}
I20260812 14:45:56.632055 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:56.622673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:33961 (local address 127.15.227.194:39075)
0812 14:45:56.622827 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 14:45:56.622836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:45:56.622864 (+ 28us) server_negotiation.cc:407] Connection header received
0812 14:45:56.623275 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:45:56.623277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:45:56.623313 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:45:56.623441 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:45:56.624167 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.625031 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.626038 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.626443 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.626554 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:56.628654 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:45:56.628659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:45:56.628659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:45:56.628670 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:45:56.630458 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:56.630843 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:56.630845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:56.630846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:56.630927 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:45:56.631151 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:45:56.631152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:45:56.631152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:45:56.631443 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:45:56.631522 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:45:56.631818 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 14:45:56.631945 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260812 14:45:56.632088 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:56.622386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:37929 (local address 127.15.227.193:60165)
0812 14:45:56.622514 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:56.622524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:45:56.622582 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:45:56.623483 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:45:56.623486 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:45:56.623849 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:45:56.624052 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:56.624058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.625123 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.625124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:56.625929 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:45:56.625932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:45:56.626530 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:45:56.626531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:45:56.626642 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:45:56.627226 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:45:56.627231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:45:56.628929 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:45:56.630785 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:56.630786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:56.630786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:56.630938 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:56.631147 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:45:56.631147 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:45:56.631148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:45:56.631199 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:45:56.631568 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:45:56.631568 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:45:56.631659 (+ 91us) client_negotiation.cc:769] Sending connection context
0812 14:45:56.631765 (+ 106us) client_negotiation.cc:240] Negotiation successful
0812 14:45:56.631984 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":164,"thread_start_us":52,"threads_started":1}
W20260812 14:45:56.633374 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
W20260812 14:45:56.633448 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
I20260812 14:45:56.633481 15503 leader_election.cc:302] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73479b3759e745aebc58100a407dc5fb; no voters: 1ea6f87083c4487485a473c99bf3d1ef, ed8e402316f042f090bbfb56846624e0
I20260812 14:45:56.633598 16219 raft_consensus.cc:2726] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:45:57.097674 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:57.097528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:49793)
0812 14:45:57.097594 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:57.097634 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 14:45:57.144170 16231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:57.144003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:40747)
0812 14:45:57.144100 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:57.144133 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":49,"threads_started":1}
I20260812 14:45:57.268975 16235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:57.268801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:50521)
0812 14:45:57.268913 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:57.268941 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260812 14:45:58.098310 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:58.098087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:49677)
0812 14:45:58.098228 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:58.098263 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260812 14:45:58.144798 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:58.144566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:58049)
0812 14:45:58.144712 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:58.144748 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260812 14:45:58.269538 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:58.269346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:38937)
0812 14:45:58.269471 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:58.269502 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260812 14:45:58.690896 16239 raft_consensus.cc:482] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:45:58.690994 16239 raft_consensus.cc:504] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:45:58.691143 16239 leader_election.cc:288] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929), 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
W20260812 14:45:58.691535 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
W20260812 14:45:58.691605 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
I20260812 14:45:58.691622 15503 leader_election.cc:302] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73479b3759e745aebc58100a407dc5fb; no voters: 1ea6f87083c4487485a473c99bf3d1ef, ed8e402316f042f090bbfb56846624e0
I20260812 14:45:58.691684 16239 raft_consensus.cc:2726] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:45:59.098868 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:59.098679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:58335)
0812 14:45:59.098787 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:59.098822 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260812 14:45:59.145393 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:59.145176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:37851)
0812 14:45:59.145304 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:59.145339 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260812 14:45:59.270103 16247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:45:59.269894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:57137)
0812 14:45:59.270025 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 14:45:59.270064 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260812 14:46:00.099517 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:00.099267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:57081)
0812 14:46:00.099434 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:00.099469 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260812 14:46:00.145964 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:00.145787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:33849)
0812 14:46:00.145894 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:00.145927 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260812 14:46:00.261402 16256 raft_consensus.cc:482] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:00.261497 16256 raft_consensus.cc:504] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:46:00.261651 16256 leader_election.cc:288] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929), 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
W20260812 14:46:00.262004 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
W20260812 14:46:00.262136 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
I20260812 14:46:00.262168 15503 leader_election.cc:302] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73479b3759e745aebc58100a407dc5fb; no voters: 1ea6f87083c4487485a473c99bf3d1ef, ed8e402316f042f090bbfb56846624e0
I20260812 14:46:00.262253 16256 raft_consensus.cc:2726] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:46:00.270609 16257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:00.270438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:51985)
0812 14:46:00.270534 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:00.270565 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260812 14:46:01.100117 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:01.099918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:39133)
0812 14:46:01.100036 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:01.100071 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260812 14:46:01.146517 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:01.146333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:43113)
0812 14:46:01.146446 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:01.146478 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260812 14:46:01.271102 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:01.270926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:59201)
0812 14:46:01.271033 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:01.271066 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":59,"threads_started":1}
I20260812 14:46:02.100816 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:02.100573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:54243)
0812 14:46:02.100723 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:02.100759 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260812 14:46:02.147080 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:02.146900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:39683)
0812 14:46:02.147008 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:02.147042 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260812 14:46:02.271636 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:02.271435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:45535)
0812 14:46:02.271548 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:02.271589 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260812 14:46:03.101450 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:03.101250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:60551)
0812 14:46:03.101368 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:03.101402 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260812 14:46:03.147676 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:03.147484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:37593)
0812 14:46:03.147603 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:03.147637 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260812 14:46:03.272200 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:03.272014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:37223)
0812 14:46:03.272131 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:03.272161 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260812 14:46:04.102067 16290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:04.101833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:44909)
0812 14:46:04.101955 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:04.101990 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260812 14:46:04.144443 16294 raft_consensus.cc:482] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:04.144522 16294 raft_consensus.cc:504] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8e402316f042f090bbfb56846624e0" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 37929 } } peers { permanent_uuid: "1ea6f87083c4487485a473c99bf3d1ef" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 39075 } } peers { permanent_uuid: "73479b3759e745aebc58100a407dc5fb" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33809 } }
I20260812 14:46:04.144652 16294 leader_election.cc:288] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929), 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075)
W20260812 14:46:04.145007 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1ea6f87083c4487485a473c99bf3d1ef (127.15.227.194:39075): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
W20260812 14:46:04.145071 15503 leader_election.cc:341] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ed8e402316f042f090bbfb56846624e0 (127.15.227.195:37929): Not found: Tablet not found: fda1420c23e64576bb798b4809e6cf12
I20260812 14:46:04.145085 15503 leader_election.cc:302] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 73479b3759e745aebc58100a407dc5fb; no voters: 1ea6f87083c4487485a473c99bf3d1ef, ed8e402316f042f090bbfb56846624e0
I20260812 14:46:04.145152 16294 raft_consensus.cc:2726] T fda1420c23e64576bb798b4809e6cf12 P 73479b3759e745aebc58100a407dc5fb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 14:46:04.148206 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:04.148018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:35305)
0812 14:46:04.148130 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:04.148162 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
I20260812 14:46:04.272846 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:04.272598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:57791)
0812 14:46:04.272753 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:04.272789 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260812 14:46:05.102650 16298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:05.102438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:50089)
0812 14:46:05.102572 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:05.102603 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260812 14:46:05.148778 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:05.148595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:35513)
0812 14:46:05.148707 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:05.148739 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260812 14:46:05.273473 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:05.273248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:35653)
0812 14:46:05.273393 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:05.273431 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260812 14:46:06.103258 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:06.103048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:60333)
0812 14:46:06.103164 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:06.103211 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260812 14:46:06.149348 16304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:06.149148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:33847)
0812 14:46:06.149274 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:06.149308 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260812 14:46:06.274149 16305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:06.273895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:41685)
0812 14:46:06.274052 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:06.274088 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260812 14:46:07.103844 16307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:07.103641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:47973)
0812 14:46:07.103765 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:07.103797 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260812 14:46:07.149902 16308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:07.149726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:46963)
0812 14:46:07.149832 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:07.149864 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260812 14:46:07.274757 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:07.274547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:52861)
0812 14:46:07.274675 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:07.274713 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260812 14:46:08.104431 16312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:08.104222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:58927)
0812 14:46:08.104348 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:08.104384 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":61,"threads_started":1}
I20260812 14:46:08.150477 16313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:08.150281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:39851)
0812 14:46:08.150405 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:08.150435 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260812 14:46:08.275384 16314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:08.275168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:46761)
0812 14:46:08.275308 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:08.275344 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260812 14:46:09.104990 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:09.104790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:38849)
0812 14:46:09.104911 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:09.104943 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260812 14:46:09.151000 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:09.150823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:34759)
0812 14:46:09.150932 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:09.150963 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
I20260812 14:46:09.275913 16318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:09.275728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:38301)
0812 14:46:09.275843 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:09.275875 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260812 14:46:10.105638 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:10.105424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.193:48703)
0812 14:46:10.105557 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:10.105590 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260812 14:46:10.151587 16325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:10.151380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.194:45211)
0812 14:46:10.151506 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:10.151541 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":54,"threads_started":1}
I20260812 14:46:10.268941 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 15401
I20260812 14:46:10.276486 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:10.276313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:33275 (local address 127.15.227.195:59649)
0812 14:46:10.276419 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:10.276445 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260812 14:46:10.279249 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 15706
I20260812 14:46:10.294287 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 15950
2026-08-12T14:46:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58456 ms)
[----------] 30 tests from SecurityITest (230282 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:10 dist-test-slave-q9h9 krb5kdc[16329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:10 dist-test-slave-q9h9 krb5kdc[16329](info): set up 2 sockets
Aug 12 14:46:10 dist-test-slave-q9h9 krb5kdc[16329](info): commencing operation
krb5kdc: starting...
W20260812 14:46:11.503398 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:11 dist-test-slave-q9h9 krb5kdc[16329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:11.610585 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:42033
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:42033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:11.690255 16353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:11.690435 16353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:11.690465 16353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:11.691553 16353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:11.691601 16353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:11.691619 16353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:11.691635 16353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:11.691650 16353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:11.693115 16353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:42033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:42033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:11.693341 16353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:11.693532 16353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:11.697921 16364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:11.697962 16360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:11.697965 16369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:11.698097 16353 server_base.cc:1034] running on GCE node
I20260812 14:46:11.698446 16353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:11.698716 16353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:11.699851 16353 hybrid_clock.cc:648] HybridClock initialized: now 1786545971699839 us; error 31 us; skew 500 ppm
Aug 12 14:46:11 dist-test-slave-q9h9 krb5kdc[16329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:11.702160 16353 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:11.702575 16353 webserver.cc:460] Webserver started at http://127.15.227.254:44687/ using document root <none> and password file <none>
I20260812 14:46:11.702800 16353 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:11.702845 16353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:11.702967 16353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:11.703905 16353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "7e250b04e97c4d82a78685f1db1832a7"
format_stamp: "Formatted at 2026-08-12 14:46:11 on dist-test-slave-q9h9"
server_key: "5c4737523fc30b8fc64ac77dc0c6fd52"
server_key_iv: "c3ec3ad18116f725b4ed6f42fda40803"
server_key_version: "encryptionkey@0"
I20260812 14:46:11.704224 16353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "7e250b04e97c4d82a78685f1db1832a7"
format_stamp: "Formatted at 2026-08-12 14:46:11 on dist-test-slave-q9h9"
server_key: "5c4737523fc30b8fc64ac77dc0c6fd52"
server_key_iv: "c3ec3ad18116f725b4ed6f42fda40803"
server_key_version: "encryptionkey@0"
I20260812 14:46:11.705443 16353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:46:11.706156 16386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:11.706305 16353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:11.706374 16353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "7e250b04e97c4d82a78685f1db1832a7"
format_stamp: "Formatted at 2026-08-12 14:46:11 on dist-test-slave-q9h9"
server_key: "5c4737523fc30b8fc64ac77dc0c6fd52"
server_key_iv: "c3ec3ad18116f725b4ed6f42fda40803"
server_key_version: "encryptionkey@0"
I20260812 14:46:11.706441 16353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:11.722113 16353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:11.723678 16353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:11.723801 16353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:11.727408 16353 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:42033
I20260812 14:46:11.727445 16489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:42033 every 8 connection(s)
I20260812 14:46:11.727727 16353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:11.728587 16490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:11.730715 16490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7: Bootstrap starting.
I20260812 14:46:11.731226 16490 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:11.731462 16490 log.cc:824] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:11.732084 16490 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7: No bootstrap required, opened a new log
I20260812 14:46:11.733608 16490 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e250b04e97c4d82a78685f1db1832a7" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 42033 } }
I20260812 14:46:11.733740 16490 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:11.733773 16490 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e250b04e97c4d82a78685f1db1832a7, State: Initialized, Role: FOLLOWER
I20260812 14:46:11.733881 16490 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e250b04e97c4d82a78685f1db1832a7" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 42033 } }
I20260812 14:46:11.733954 16490 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:11.733981 16490 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:11.734040 16490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:11.734513 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 16353
I20260812 14:46:11.734540 16490 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e250b04e97c4d82a78685f1db1832a7" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 42033 } }
I20260812 14:46:11.734608 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:11.734620 16490 leader_election.cc:302] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e250b04e97c4d82a78685f1db1832a7; no voters:
I20260812 14:46:11.734774 16490 leader_election.cc:288] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:11.734814 16271 external_mini_cluster.cc:1428] Setting key 766d1d7815e921a5ec60ed57eaecd778
I20260812 14:46:11.734824 16497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:11.734969 16497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [term 1 LEADER]: Becoming Leader. State: Replica: 7e250b04e97c4d82a78685f1db1832a7, State: Running, Role: LEADER
I20260812 14:46:11.735049 16490 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:11.735136 16497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e250b04e97c4d82a78685f1db1832a7" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 42033 } }
I20260812 14:46:11.735455 16497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e250b04e97c4d82a78685f1db1832a7. Latest consensus state: current_term: 1 leader_uuid: "7e250b04e97c4d82a78685f1db1832a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e250b04e97c4d82a78685f1db1832a7" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 42033 } } }
I20260812 14:46:11.735534 16497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:11.735605 16503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e250b04e97c4d82a78685f1db1832a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e250b04e97c4d82a78685f1db1832a7" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 42033 } } }
I20260812 14:46:11.735685 16503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:11.736151 16513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:11.736749 16513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:11.738323 16513 catalog_manager.cc:1347] Generated new cluster ID: 5aa5b754101f4cafb6100033a9e1e4e3
I20260812 14:46:11.738379 16513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:46:11 dist-test-slave-q9h9 krb5kdc[16329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:11.745800 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:11.735387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56238 (local address 127.15.227.254:42033)
0812 14:46:11.735578 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:11.735588 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:11.735618 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:46:11.735723 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:11.735730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:11.735796 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:11.735927 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:11.736673 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:11.737919 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:11.738995 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:11.739594 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:11.739675 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:11.742255 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:11.742259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:11.742259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:11.742279 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:11.744022 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:11.744374 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:11.744377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:11.744377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:11.744482 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:11.744640 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:11.744641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:11.744641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:11.744913 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:11.745043 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:11.745473 (+ 430us) server_negotiation.cc:299] Negotiation successful
0812 14:46:11.745592 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260812 14:46:11.770009 16513 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:11.770502 16513 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:11.773031 16513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7: Generated new TSK 0
I20260812 14:46:11.773166 16513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:46:11.804224 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:42033
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:11.900595 16545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:11.900781 16545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:11.900803 16545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:11.901902 16545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:11.901940 16545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:11.901965 16545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:46:11.903416 16545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:42033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:11.903625 16545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:11.903831 16545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:11.908517 16553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:11.908567 16552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:11.908675 16545 server_base.cc:1034] running on GCE node
W20260812 14:46:11.908519 16555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:11.908973 16545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:11.909207 16545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:11.910363 16545 hybrid_clock.cc:648] HybridClock initialized: now 1786545971910351 us; error 31 us; skew 500 ppm
Aug 12 14:46:11 dist-test-slave-q9h9 krb5kdc[16329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:11.912596 16545 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:46:11.912981 16545 webserver.cc:460] Webserver started at http://127.15.227.193:46657/ using document root <none> and password file <none>
I20260812 14:46:11.913169 16545 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:11.913214 16545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:11.913340 16545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:11.914326 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "1b551e56d4b742ba96a26ffcba46b4dc"
format_stamp: "Formatted at 2026-08-12 14:46:11 on dist-test-slave-q9h9"
server_key: "2b9a600784a2fbc0cf0850117b920cd6"
server_key_iv: "361e3654c8381a8bbf36e028b066c1cc"
server_key_version: "encryptionkey@0"
I20260812 14:46:11.914633 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "1b551e56d4b742ba96a26ffcba46b4dc"
format_stamp: "Formatted at 2026-08-12 14:46:11 on dist-test-slave-q9h9"
server_key: "2b9a600784a2fbc0cf0850117b920cd6"
server_key_iv: "361e3654c8381a8bbf36e028b066c1cc"
server_key_version: "encryptionkey@0"
I20260812 14:46:11.915822 16545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:11.916417 16566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:11.916568 16545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:11.916649 16545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "1b551e56d4b742ba96a26ffcba46b4dc"
format_stamp: "Formatted at 2026-08-12 14:46:11 on dist-test-slave-q9h9"
server_key: "2b9a600784a2fbc0cf0850117b920cd6"
server_key_iv: "361e3654c8381a8bbf36e028b066c1cc"
server_key_version: "encryptionkey@0"
I20260812 14:46:11.916705 16545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:11.925640 16545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:11.927277 16545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:11.927403 16545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:11.927589 16545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:11.927858 16545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:11.927899 16545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:11.927925 16545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:11.927942 16545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:11.934163 16545 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:44351
I20260812 14:46:11.934221 16793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:44351 every 8 connection(s)
I20260812 14:46:11.934495 16545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:46:11.938983 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 16545
I20260812 14:46:11.939097 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:46:11.939291 16271 external_mini_cluster.cc:1428] Setting key 01b04a2dae88d1eae5227a3b51b826fc
Aug 12 14:46:11 dist-test-slave-q9h9 krb5kdc[16329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545971, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:11.943174 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:11.935142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:32877 (local address 127.15.227.254:42033)
0812 14:46:11.935213 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:11.935215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:11.935368 (+ 153us) server_negotiation.cc:407] Connection header received
0812 14:46:11.935787 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:11.935789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:11.935799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:11.935849 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:11.936708 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:11.937261 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:11.938204 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:11.938406 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:11.938453 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:11.940633 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:11.940634 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:11.940634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:11.940640 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:11.941923 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:11.942324 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:11.942325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:11.942326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:11.942393 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:11.942591 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:11.942592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:11.942593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:11.942730 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:11.942801 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:11.943005 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 14:46:11.943088 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:46:11.943216 16796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:11.935101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:42033 (local address 127.15.227.193:32877)
0812 14:46:11.935250 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:11.935289 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:11.935708 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:11.935909 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:11.935913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:11.936455 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:11.936629 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:11.936634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:11.937332 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:11.937332 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:11.938132 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:11.938135 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:11.938461 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:11.938461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:11.938556 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:11.938964 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:11.938972 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:11.940548 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:11.942000 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:11.942007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:11.942009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:11.942250 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:11.942454 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:11.942454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:11.942454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:11.942542 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:11.942790 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:11.942793 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:11.942892 (+ 99us) client_negotiation.cc:769] Sending connection context
0812 14:46:11.942964 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:46:11.943072 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260812 14:46:11.943853 16794 heartbeater.cc:347] Connected to a master server at 127.15.227.254:42033
I20260812 14:46:11.943938 16794 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:11.944096 16794 heartbeater.cc:511] Master 127.15.227.254:42033 requested a full tablet report, sending...
I20260812 14:46:11.944578 16415 ts_manager.cc:194] Registered new tserver with Master: 1b551e56d4b742ba96a26ffcba46b4dc (127.15.227.193:44351)
I20260812 14:46:11.945154 16415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:32877
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:46:11.962385 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:42033
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:12.042501 16799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:12.042693 16799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:12.042716 16799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:12.043833 16799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:12.043879 16799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:12.043910 16799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:46:12.045349 16799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:42033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:12.045570 16799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:12.045796 16799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:12.050518 16809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.050572 16816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.050627 16808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:12.050710 16799 server_base.cc:1034] running on GCE node
I20260812 14:46:12.050885 16799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:12.051105 16799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:12.052245 16799 hybrid_clock.cc:648] HybridClock initialized: now 1786545972052228 us; error 32 us; skew 500 ppm
Aug 12 14:46:12 dist-test-slave-q9h9 krb5kdc[16329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545972, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:12.054613 16799 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:46:12.055002 16799 webserver.cc:460] Webserver started at http://127.15.227.194:41869/ using document root <none> and password file <none>
I20260812 14:46:12.055208 16799 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:12.055262 16799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:12.055387 16799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:12.056351 16799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "1cf269e8c1eb4dda9e008393a93e1362"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "011667cdca32f9813907930cf714a373"
server_key_iv: "a9f5051ca058cdeaddfd2800f205e368"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.056655 16799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "1cf269e8c1eb4dda9e008393a93e1362"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "011667cdca32f9813907930cf714a373"
server_key_iv: "a9f5051ca058cdeaddfd2800f205e368"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.057828 16799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:46:12.058548 16830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.058712 16799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.058794 16799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "1cf269e8c1eb4dda9e008393a93e1362"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "011667cdca32f9813907930cf714a373"
server_key_iv: "a9f5051ca058cdeaddfd2800f205e368"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.058853 16799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:12.067782 16799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:12.069321 16799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:12.069440 16799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:12.069615 16799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:12.069864 16799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:12.069896 16799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.069921 16799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:12.069947 16799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.075268 16799 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:34699
I20260812 14:46:12.075307 17032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:34699 every 8 connection(s)
I20260812 14:46:12.075565 16799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:46:12.076284 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 16799
I20260812 14:46:12.076417 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:46:12.076604 16271 external_mini_cluster.cc:1428] Setting key 2b3c4de7e018d3ab132db926dd3e8959
Aug 12 14:46:12 dist-test-slave-q9h9 krb5kdc[16329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545972, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:12.083928 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.076192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:57851 (local address 127.15.227.254:42033)
0812 14:46:12.076238 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.076240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.076441 (+ 201us) server_negotiation.cc:407] Connection header received
0812 14:46:12.076897 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.076898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.076911 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.076962 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:12.077809 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.078343 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.079286 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.079480 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.079534 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.081403 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:12.081405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:12.081405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:12.081413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:12.082729 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:12.083083 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:12.083084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:12.083084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:12.083135 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:12.083345 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:12.083345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:12.083346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:12.083449 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:12.083505 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.083750 (+ 245us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.083840 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:46:12.084048 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.076178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:42033 (local address 127.15.227.194:57851)
0812 14:46:12.076328 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.076366 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.076797 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.077025 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.077029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.077541 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:12.077713 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.077717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.078426 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.078428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.079201 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.079203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.079547 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.079548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.079668 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.080090 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:12.080098 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:12.081318 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:12.082798 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:12.082805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:12.082807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:12.083009 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:12.083191 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:12.083191 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:12.083192 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:12.083290 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:12.083505 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:12.083513 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:12.083636 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.083720 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.083858 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260812 14:46:12.084431 17033 heartbeater.cc:347] Connected to a master server at 127.15.227.254:42033
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
I20260812 14:46:12.084512 17033 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:12.084662 17033 heartbeater.cc:511] Master 127.15.227.254:42033 requested a full tablet report, sending...
I20260812 14:46:12.085008 16415 ts_manager.cc:194] Registered new tserver with Master: 1cf269e8c1eb4dda9e008393a93e1362 (127.15.227.194:34699)
I20260812 14:46:12.085389 16415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:57851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:46:12.098915 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:42033
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:12.177160 17046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:12.177335 17046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:12.177356 17046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:12.178481 17046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:12.178522 17046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:12.178547 17046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:46:12.179944 17046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:42033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:12.180154 17046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:12.180349 17046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:12.184928 17057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.184988 17067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.184968 17061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:12.185148 17046 server_base.cc:1034] running on GCE node
I20260812 14:46:12.185341 17046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:12.185525 17046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:12.186647 17046 hybrid_clock.cc:648] HybridClock initialized: now 1786545972186635 us; error 27 us; skew 500 ppm
Aug 12 14:46:12 dist-test-slave-q9h9 krb5kdc[16329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545972, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:12.188895 17046 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:46:12.189271 17046 webserver.cc:460] Webserver started at http://127.15.227.195:33819/ using document root <none> and password file <none>
I20260812 14:46:12.189455 17046 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:12.189500 17046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:12.189618 17046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:12.190580 17046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "415f1c00da1e4393a0a8a160363db17d"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "1e205cdb3a80768ec1ffe42b1db44be8"
server_key_iv: "8ee131733357e9cf09cc3c5ec0b229c6"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.190884 17046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "415f1c00da1e4393a0a8a160363db17d"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "1e205cdb3a80768ec1ffe42b1db44be8"
server_key_iv: "8ee131733357e9cf09cc3c5ec0b229c6"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.192025 17046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:12.192682 17081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.192848 17046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:12.192942 17046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "415f1c00da1e4393a0a8a160363db17d"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "1e205cdb3a80768ec1ffe42b1db44be8"
server_key_iv: "8ee131733357e9cf09cc3c5ec0b229c6"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.193014 17046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:12.203169 17046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:12.205018 17046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:12.205117 17046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:12.205281 17046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:12.205504 17046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:12.205533 17046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.205554 17046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:12.205569 17046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.211004 17046 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:35671
I20260812 14:46:12.211045 17303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:35671 every 8 connection(s)
I20260812 14:46:12.211295 17046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:46:12.212790 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 17046
I20260812 14:46:12.212929 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:46:12.213126 16271 external_mini_cluster.cc:1428] Setting key 340a76f110aa5ca4ebd5ce01379e61c2
Aug 12 14:46:12 dist-test-slave-q9h9 krb5kdc[16329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545972, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:12.219374 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.211863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:56321 (local address 127.15.227.254:42033)
0812 14:46:12.211934 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.211936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.212096 (+ 160us) server_negotiation.cc:407] Connection header received
0812 14:46:12.212513 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.212514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.212523 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.212566 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:12.213700 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.214329 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.215223 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.215398 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.215440 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.217268 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:12.217269 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:12.217270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:12.217276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:12.218349 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:12.218678 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:12.218678 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:12.218679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:12.218722 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:12.218889 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:12.218889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:12.218889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:12.218977 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:12.219020 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.219246 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.219303 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 14:46:12.219578 17310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.211840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:42033 (local address 127.15.227.195:56321)
0812 14:46:12.211983 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.212014 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.212438 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.212649 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.212654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.213351 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:12.213598 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.213605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.214396 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.214397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.215161 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.215163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.215447 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.215448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.215537 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.215950 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:12.215958 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:12.217191 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:12.218408 (+ 1217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:12.218413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:12.218415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:12.218613 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:12.218764 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:12.218764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:12.218764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:12.218847 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:12.219025 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:12.219029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:12.219152 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.219229 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.219381 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":74,"threads_started":1}
I20260812 14:46:12.219981 17306 heartbeater.cc:347] Connected to a master server at 127.15.227.254:42033
I20260812 14:46:12.220077 17306 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:12.220237 17306 heartbeater.cc:511] Master 127.15.227.254:42033 requested a full tablet report, sending...
I20260812 14:46:12.220582 16415 ts_manager.cc:194] Registered new tserver with Master: 415f1c00da1e4393a0a8a160363db17d (127.15.227.195:35671)
I20260812 14:46:12.220932 16415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:56321
I20260812 14:46:12.224524 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:46:12.230726 16507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.225792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56254 (local address 127.15.227.254:42033)
0812 14:46:12.225837 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.225838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.225943 (+ 105us) server_negotiation.cc:407] Connection header received
0812 14:46:12.225988 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.225988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.225996 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.226069 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:12.226568 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.227005 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.227661 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.227837 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.227886 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.228889 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:12.228891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:12.228891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:12.228897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:12.229888 (+ 991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:12.230204 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:12.230205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:12.230205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:12.230252 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:12.230374 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:12.230374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:12.230374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:12.230487 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:12.230554 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.230618 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.230671 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:46:12.232082 16415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:46:12.233568 16415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:46:12.239413 17322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.235452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44351 (local address 127.0.0.1:43640)
0812 14:46:12.235592 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.235626 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.235686 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.236120 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.236123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.236125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:12.236181 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.236183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.237213 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.237213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.238487 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.238490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.239139 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.239140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.239216 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.239229 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.239291 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.239342 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":26,"thread_start_us":47,"threads_started":1}
I20260812 14:46:12.239425 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.235604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.15.227.193:44351)
0812 14:46:12.235965 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.235970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.235980 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:46:12.236018 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.236020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.236035 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.236107 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:12.236242 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.237122 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.238582 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.239064 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.239130 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.239181 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.239311 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.239351 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":88,"threads_started":1}
I20260812 14:46:12.239670 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.235452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35671 (local address 127.0.0.1:45580)
0812 14:46:12.235592 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.235628 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.235686 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.236170 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.236171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.236173 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:12.236227 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.236229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.237444 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.237445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.238547 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.238549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.239420 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.239421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.239489 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.239493 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.239539 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.239579 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":22,"thread_start_us":46,"threads_started":1}
I20260812 14:46:12.239739 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.235710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.15.227.194:34699)
0812 14:46:12.235904 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.235909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.235946 (+ 37us) server_negotiation.cc:407] Connection header received
0812 14:46:12.236026 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.236027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.236041 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.236132 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:12.236315 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.237514 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.238556 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.239267 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.239335 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.239383 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.239623 (+ 240us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.239664 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260812 14:46:12.239868 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.235494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34699 (local address 127.0.0.1:40012)
0812 14:46:12.235882 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.235892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.235958 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.236206 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.236207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.236209 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:12.236240 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.236241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.237598 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.237599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.238485 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.238488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.239421 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.239421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.239489 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.239493 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.239539 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.239579 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260812 14:46:12.240060 17330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.235798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.15.227.195:35671)
0812 14:46:12.235986 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.235989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.235997 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:12.236024 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.236026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.236040 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.236113 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:12.236360 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.237347 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.238605 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.239266 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.239335 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.239383 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.239948 (+ 565us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.239993 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":68,"threads_started":1}
I20260812 14:46:12.240213 16647 tablet_service.cc:1467] Processing CreateTablet for tablet 0095d049b310451da73922278bf07398 (DEFAULT_TABLE table=test-table [id=60324360eb104a1d98e837c8509a60df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:12.240569 16647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0095d049b310451da73922278bf07398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:12.240651 17176 tablet_service.cc:1467] Processing CreateTablet for tablet 0095d049b310451da73922278bf07398 (DEFAULT_TABLE table=test-table [id=60324360eb104a1d98e837c8509a60df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:12.240963 17176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0095d049b310451da73922278bf07398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:12.243065 17331 tablet_bootstrap.cc:492] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc: Bootstrap starting.
I20260812 14:46:12.243417 17333 tablet_bootstrap.cc:492] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Bootstrap starting.
I20260812 14:46:12.243678 16916 tablet_service.cc:1467] Processing CreateTablet for tablet 0095d049b310451da73922278bf07398 (DEFAULT_TABLE table=test-table [id=60324360eb104a1d98e837c8509a60df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:12.243956 17331 tablet_bootstrap.cc:654] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:12.243979 17333 tablet_bootstrap.cc:654] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:12.243992 16916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0095d049b310451da73922278bf07398. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:12.244361 17333 log.cc:824] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:12.244464 17331 log.cc:824] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:12.245054 17333 tablet_bootstrap.cc:492] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: No bootstrap required, opened a new log
I20260812 14:46:12.245090 17331 tablet_bootstrap.cc:492] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc: No bootstrap required, opened a new log
I20260812 14:46:12.245113 17333 ts_tablet_manager.cc:1397] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:12.245136 17331 ts_tablet_manager.cc:1397] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:12.246524 17331 raft_consensus.cc:348] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.246654 17331 raft_consensus.cc:374] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:12.246704 17331 raft_consensus.cc:729] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b551e56d4b742ba96a26ffcba46b4dc, State: Initialized, Role: FOLLOWER
I20260812 14:46:12.246809 17331 consensus_queue.cc:260] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.247032 17333 raft_consensus.cc:348] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.247179 17331 ts_tablet_manager.cc:1428] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:46:12.247195 17333 raft_consensus.cc:374] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:12.247231 17333 raft_consensus.cc:729] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 415f1c00da1e4393a0a8a160363db17d, State: Initialized, Role: FOLLOWER
I20260812 14:46:12.247341 17333 consensus_queue.cc:260] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.247478 17342 tablet_bootstrap.cc:492] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: Bootstrap starting.
I20260812 14:46:12.247609 17333 ts_tablet_manager.cc:1428] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:46:12.247620 17306 heartbeater.cc:503] Master 127.15.227.254:42033 was elected leader, sending a full tablet report...
I20260812 14:46:12.247758 16794 heartbeater.cc:503] Master 127.15.227.254:42033 was elected leader, sending a full tablet report...
I20260812 14:46:12.248083 17342 tablet_bootstrap.cc:654] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:12.248631 17342 log.cc:824] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:12.249313 17342 tablet_bootstrap.cc:492] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: No bootstrap required, opened a new log
I20260812 14:46:12.249368 17342 ts_tablet_manager.cc:1397] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:12.250684 17342 raft_consensus.cc:348] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.250808 17342 raft_consensus.cc:374] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:12.250841 17342 raft_consensus.cc:729] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cf269e8c1eb4dda9e008393a93e1362, State: Initialized, Role: FOLLOWER
I20260812 14:46:12.250983 17342 consensus_queue.cc:260] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.251284 17342 ts_tablet_manager.cc:1428] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:46:12.251329 17033 heartbeater.cc:503] Master 127.15.227.254:42033 was elected leader, sending a full tablet report...
W20260812 14:46:12.326086 17034 tablet.cc:2369] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:46:12.375702 17346 raft_consensus.cc:482] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:12.375795 17346 raft_consensus.cc:504] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.376082 17346 leader_election.cc:288] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b551e56d4b742ba96a26ffcba46b4dc (127.15.227.193:44351), 415f1c00da1e4393a0a8a160363db17d (127.15.227.195:35671)
I20260812 14:46:12.378867 17330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.376358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:51017 (local address 127.15.227.195:35671)
0812 14:46:12.376403 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.376404 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.376412 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:12.376436 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.376437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.376446 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.376490 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:12.376640 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.377133 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.378165 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.378554 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.378616 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.378625 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.378773 (+ 148us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.378806 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:46:12.378906 17326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.376361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:43193 (local address 127.15.227.193:44351)
0812 14:46:12.376404 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.376406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.376494 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:46:12.376544 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.376545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.376553 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.376598 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:12.376724 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.377163 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.378240 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.378602 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.378664 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.378673 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.378810 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.378843 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:46:12.378902 17036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.376244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:35671 (local address 127.15.227.194:51017)
0812 14:46:12.376297 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.376308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.376352 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.376502 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.376503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.376505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:12.376576 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.376578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.377211 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.377212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.378103 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.378105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.378633 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.378635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.378714 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.378717 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.378761 (+ 44us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.378800 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":17}
I20260812 14:46:12.378997 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.376251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44351 (local address 127.15.227.194:43193)
0812 14:46:12.376432 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.376440 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.376495 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.376607 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.376608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.376609 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:12.376672 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.376674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.377233 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.377234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.378161 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.378163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.378684 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.378685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.378756 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.378759 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.378801 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.378839 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":14,"thread_start_us":105,"threads_started":1}
I20260812 14:46:12.379240 17213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0095d049b310451da73922278bf07398" candidate_uuid: "1cf269e8c1eb4dda9e008393a93e1362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "415f1c00da1e4393a0a8a160363db17d" is_pre_election: true
I20260812 14:46:12.379278 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0095d049b310451da73922278bf07398" candidate_uuid: "1cf269e8c1eb4dda9e008393a93e1362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" is_pre_election: true
I20260812 14:46:12.379375 17213 raft_consensus.cc:2454] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf269e8c1eb4dda9e008393a93e1362 in term 0.
I20260812 14:46:12.379397 16712 raft_consensus.cc:2454] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cf269e8c1eb4dda9e008393a93e1362 in term 0.
I20260812 14:46:12.379592 16835 leader_election.cc:302] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b551e56d4b742ba96a26ffcba46b4dc, 1cf269e8c1eb4dda9e008393a93e1362; no voters:
I20260812 14:46:12.379719 17346 raft_consensus.cc:2781] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:46:12.379773 17346 raft_consensus.cc:482] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:46:12.379789 17346 raft_consensus.cc:3037] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:12.380463 17346 raft_consensus.cc:504] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.380581 17346 leader_election.cc:288] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [CANDIDATE]: Term 1 election: Requested vote from peers 1b551e56d4b742ba96a26ffcba46b4dc (127.15.227.193:44351), 415f1c00da1e4393a0a8a160363db17d (127.15.227.195:35671)
I20260812 14:46:12.380733 17213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0095d049b310451da73922278bf07398" candidate_uuid: "1cf269e8c1eb4dda9e008393a93e1362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "415f1c00da1e4393a0a8a160363db17d"
I20260812 14:46:12.380750 16712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0095d049b310451da73922278bf07398" candidate_uuid: "1cf269e8c1eb4dda9e008393a93e1362" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b551e56d4b742ba96a26ffcba46b4dc"
I20260812 14:46:12.380793 17213 raft_consensus.cc:3037] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:12.380817 16712 raft_consensus.cc:3037] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:12.381455 17213 raft_consensus.cc:2454] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf269e8c1eb4dda9e008393a93e1362 in term 1.
I20260812 14:46:12.381503 16712 raft_consensus.cc:2454] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cf269e8c1eb4dda9e008393a93e1362 in term 1.
I20260812 14:46:12.381629 16832 leader_election.cc:302] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cf269e8c1eb4dda9e008393a93e1362, 415f1c00da1e4393a0a8a160363db17d; no voters:
I20260812 14:46:12.381724 17346 raft_consensus.cc:2781] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:12.381842 17346 raft_consensus.cc:686] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 1 LEADER]: Becoming Leader. State: Replica: 1cf269e8c1eb4dda9e008393a93e1362, State: Running, Role: LEADER
I20260812 14:46:12.381927 17346 consensus_queue.cc:237] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } }
I20260812 14:46:12.382691 16411 catalog_manager.cc:5495] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cf269e8c1eb4dda9e008393a93e1362 (127.15.227.194). New cstate: current_term: 1 leader_uuid: "1cf269e8c1eb4dda9e008393a93e1362" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cf269e8c1eb4dda9e008393a93e1362" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 34699 } health_report { overall_health: HEALTHY } } }
I20260812 14:46:12.417755 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.415705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40024 (local address 127.15.227.194:34699)
0812 14:46:12.415748 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.415750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.415757 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:12.415783 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.415784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.415798 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.415849 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:12.415947 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.416390 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.417094 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.417276 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.417328 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.417472 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:12.417557 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:12.417624 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.417676 (+ 52us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.417694 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:46:12.419343 17213 raft_consensus.cc:1261] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 1 FOLLOWER]: Refusing update from remote peer 1cf269e8c1eb4dda9e008393a93e1362: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:12.419344 16712 raft_consensus.cc:1261] T 0095d049b310451da73922278bf07398 P 1b551e56d4b742ba96a26ffcba46b4dc [term 1 FOLLOWER]: Refusing update from remote peer 1cf269e8c1eb4dda9e008393a93e1362: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:12.419629 17346 consensus_queue.cc:1035] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b551e56d4b742ba96a26ffcba46b4dc" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:12.419764 17351 consensus_queue.cc:1035] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [LEADER]: Connected to new peer: Peer: permanent_uuid: "415f1c00da1e4393a0a8a160363db17d" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 35671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:12.421907 17354 mvcc.cc:205] Tried to move back new op lower bound from 7317692303027421184 to 7317692302877024256. Current Snapshot: MvccSnapshot[applied={T|T < 7317692303027421184}]
I20260812 14:46:12.422149 17355 mvcc.cc:205] Tried to move back new op lower bound from 7317692303027421184 to 7317692302877024256. Current Snapshot: MvccSnapshot[applied={T|T < 7317692303027421184}]
I20260812 14:46:12.422992 17357 mvcc.cc:205] Tried to move back new op lower bound from 7317692303027421184 to 7317692302877024256. Current Snapshot: MvccSnapshot[applied={T|T < 7317692303027421184}]
I20260812 14:46:12.424141 16415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:12.424216 16415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:12.424861 16415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e250b04e97c4d82a78685f1db1832a7: Sending DeleteTablet for 3 replicas of tablet 0095d049b310451da73922278bf07398
I20260812 14:46:12.425165 16647 tablet_service.cc:1514] Processing DeleteTablet for tablet 0095d049b310451da73922278bf07398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:12 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:43640
I20260812 14:46:12.425179 17176 tablet_service.cc:1514] Processing DeleteTablet for tablet 0095d049b310451da73922278bf07398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:12 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:45580
I20260812 14:46:12.425392 17374 tablet_replica.cc:330] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: stopping tablet replica
I20260812 14:46:12.425505 17374 raft_consensus.cc:2229] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:12.425487 16916 tablet_service.cc:1514] Processing DeleteTablet for tablet 0095d049b310451da73922278bf07398 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:12 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:40012
I20260812 14:46:12.425516 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 16545
I20260812 14:46:12.425633 17374 raft_consensus.cc:2258] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:12.433139 17374 ts_tablet_manager.cc:1905] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:12.434778 17374 ts_tablet_manager.cc:1918] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:12.434892 17374 log.cc:1196] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/0095d049b310451da73922278bf07398
I20260812 14:46:12.435190 17374 ts_tablet_manager.cc:1939] T 0095d049b310451da73922278bf07398 P 415f1c00da1e4393a0a8a160363db17d: Deleting consensus metadata
I20260812 14:46:12.435679 16387 catalog_manager.cc:4841] TS 415f1c00da1e4393a0a8a160363db17d (127.15.227.195:35671): tablet 0095d049b310451da73922278bf07398 (table test-table [id=60324360eb104a1d98e837c8509a60df]) successfully deleted
I20260812 14:46:12.439530 17376 tablet_replica.cc:330] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362: stopping tablet replica
W20260812 14:46:12.439533 16389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:44351 (error 108)
I20260812 14:46:12.439677 17376 raft_consensus.cc:2229] T 0095d049b310451da73922278bf07398 P 1cf269e8c1eb4dda9e008393a93e1362 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:46:12.439743 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 16799
I20260812 14:46:12.460150 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.439939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:44351 (local address 127.0.0.1:43646)
0812 14:46:12.460056 (+ 20117us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.460112 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20090}
W20260812 14:46:12.461329 16389 catalog_manager.cc:4584] TS 1b551e56d4b742ba96a26ffcba46b4dc (127.15.227.193:44351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0095d049b310451da73922278bf07398: Network error: Client connection negotiation failed: client connection to 127.15.227.193:44351: connect: Connection refused (error 111)
I20260812 14:46:12.465061 17325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.464954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:34699 (local address 127.0.0.1:40034)
0812 14:46:12.465010 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.465038 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.194:34699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260812 14:46:12.465071 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 17046
I20260812 14:46:12.478515 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 16353
2026-08-12T14:46:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2182 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T14:46:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:12Z Disabled control of system clock
I20260812 14:46:12.511363 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34591
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:34591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:12.588915 17383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:12.589111 17383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:12.589130 17383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:12.590231 17383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:12.590265 17383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:12.590276 17383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:12.590287 17383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:12.590298 17383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:12.591572 17383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:34591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:12.591756 17383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:12.591979 17383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:12.596467 17404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.596489 17398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:12.596634 17383 server_base.cc:1034] running on GCE node
W20260812 14:46:12.596493 17399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:12.596909 17383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:12.597167 17383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:12.598301 17383 hybrid_clock.cc:648] HybridClock initialized: now 1786545972598278 us; error 36 us; skew 500 ppm
I20260812 14:46:12.599433 17383 webserver.cc:460] Webserver started at http://127.15.227.254:44621/ using document root <none> and password file <none>
I20260812 14:46:12.599704 17383 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:12.599769 17383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:12.599969 17383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:12.600919 17383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "052f1807bf884238a7c6ec38b3897386"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "bec7729418d23e1b603f21befc4aa7ca"
server_key_iv: "1003bf69b4152c1f5ea19cb9e9c48634"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.601233 17383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "052f1807bf884238a7c6ec38b3897386"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "bec7729418d23e1b603f21befc4aa7ca"
server_key_iv: "1003bf69b4152c1f5ea19cb9e9c48634"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.602494 17383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:46:12.603111 17415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.603260 17383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:12.603339 17383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "052f1807bf884238a7c6ec38b3897386"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "bec7729418d23e1b603f21befc4aa7ca"
server_key_iv: "1003bf69b4152c1f5ea19cb9e9c48634"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.603396 17383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:12.635706 17383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:12.635972 17383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:12.636096 17383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:12.639811 17383 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:34591
I20260812 14:46:12.639844 17514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:34591 every 8 connection(s)
I20260812 14:46:12.640123 17383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:12.641001 17515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:12.643157 17515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386: Bootstrap starting.
I20260812 14:46:12.643697 17515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:12.643957 17515 log.cc:824] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:12.644557 17515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386: No bootstrap required, opened a new log
I20260812 14:46:12.645440 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 17383
I20260812 14:46:12.645548 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:12.645731 16271 external_mini_cluster.cc:1428] Setting key 94ed58be32f814314a150b94d6608de0
I20260812 14:46:12.646320 17515 raft_consensus.cc:348] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052f1807bf884238a7c6ec38b3897386" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34591 } }
I20260812 14:46:12.646488 17515 raft_consensus.cc:374] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:12.646529 17515 raft_consensus.cc:729] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 052f1807bf884238a7c6ec38b3897386, State: Initialized, Role: FOLLOWER
I20260812 14:46:12.646654 17515 consensus_queue.cc:260] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052f1807bf884238a7c6ec38b3897386" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34591 } }
I20260812 14:46:12.646731 17515 raft_consensus.cc:388] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:12.646817 17515 raft_consensus.cc:482] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:12.646871 17515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:12.647363 17515 raft_consensus.cc:504] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052f1807bf884238a7c6ec38b3897386" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34591 } }
I20260812 14:46:12.647450 17515 leader_election.cc:302] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 052f1807bf884238a7c6ec38b3897386; no voters:
I20260812 14:46:12.647610 17515 leader_election.cc:288] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:12.647647 17528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:12.647811 17528 raft_consensus.cc:686] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [term 1 LEADER]: Becoming Leader. State: Replica: 052f1807bf884238a7c6ec38b3897386, State: Running, Role: LEADER
I20260812 14:46:12.647856 17515 sys_catalog.cc:564] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:12.647957 17528 consensus_queue.cc:237] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052f1807bf884238a7c6ec38b3897386" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34591 } }
I20260812 14:46:12.648306 17528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 052f1807bf884238a7c6ec38b3897386. Latest consensus state: current_term: 1 leader_uuid: "052f1807bf884238a7c6ec38b3897386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052f1807bf884238a7c6ec38b3897386" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34591 } } }
I20260812 14:46:12.648316 17538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "052f1807bf884238a7c6ec38b3897386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "052f1807bf884238a7c6ec38b3897386" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34591 } } }
I20260812 14:46:12.648391 17528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:12.648415 17538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:12.648584 17547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:12.649084 17547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:12.650981 17547 catalog_manager.cc:1347] Generated new cluster ID: 27ceb83154474c8383d142a5ba8f39c0
I20260812 14:46:12.651080 17547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:12.651386 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.646196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.15.227.254:34591)
0812 14:46:12.646417 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.646426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.646462 (+ 36us) server_negotiation.cc:407] Connection header received
0812 14:46:12.646568 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.646575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.646640 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.646777 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:12.646993 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.648480 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.649563 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.649987 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.650137 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.650727 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:12.650731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:12.650731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:12.650920 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:12.650960 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:12.650976 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:12.651039 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.651111 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.651196 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260812 14:46:12.657371 17547 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:12.658001 17547 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:12.662624 17547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386: Generated new TSK 0
I20260812 14:46:12.662768 17547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:12.670455 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34591
--builtin_ntp_servers=127.15.227.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:12.750396 17567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:12.750582 17567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:12.750612 17567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:12.751693 17567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:12.751729 17567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:12.751762 17567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:46:12.753062 17567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:12.753271 17567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:12.753520 17567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:12.759040 17580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.759057 17581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.759056 17585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:12.759248 17567 server_base.cc:1034] running on GCE node
I20260812 14:46:12.759513 17567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:12.759728 17567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:12.760864 17567 hybrid_clock.cc:648] HybridClock initialized: now 1786545972760851 us; error 31 us; skew 500 ppm
I20260812 14:46:12.762005 17567 webserver.cc:460] Webserver started at http://127.15.227.193:34207/ using document root <none> and password file <none>
I20260812 14:46:12.762271 17567 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:12.762315 17567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:12.762420 17567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:12.763381 17567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "bfd5338515c043a2b53c00a4fe2fa86f"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "8e8ff535c8c7631d27933910434d6a51"
server_key_iv: "2df28e5af4c803c1582db4e31a7883e4"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.763688 17567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "bfd5338515c043a2b53c00a4fe2fa86f"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "8e8ff535c8c7631d27933910434d6a51"
server_key_iv: "2df28e5af4c803c1582db4e31a7883e4"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.764997 17567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:46:12.765715 17594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.765882 17567 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.765965 17567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "bfd5338515c043a2b53c00a4fe2fa86f"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "8e8ff535c8c7631d27933910434d6a51"
server_key_iv: "2df28e5af4c803c1582db4e31a7883e4"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.766059 17567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:12.788884 17567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:12.789119 17567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:12.789242 17567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:12.789428 17567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:12.789709 17567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:12.789743 17567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.789769 17567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:12.789794 17567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.795364 17567 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:37639
I20260812 14:46:12.795444 17825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:37639 every 8 connection(s)
I20260812 14:46:12.795696 17567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:46:12.800391 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.796333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:34979 (local address 127.15.227.254:34591)
0812 14:46:12.796374 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.796376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.796593 (+ 217us) server_negotiation.cc:407] Connection header received
0812 14:46:12.797045 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.797047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.797061 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.797114 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:12.797736 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.798263 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.799188 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.799376 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.799433 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.800144 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:12.800146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:12.800146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:12.800151 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:12.800165 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:12.800169 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:12.800211 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.800286 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.800315 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:46:12.800578 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.796305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:34591 (local address 127.15.227.193:34979)
0812 14:46:12.796493 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.796523 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.796960 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.797183 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.797187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.797197 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:12.797652 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.797656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.798351 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.798353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.799107 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.799110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.799447 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.799448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.799578 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.800011 (+ 433us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:12.800011 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:12.800036 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:12.800058 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:12.800217 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:12.800222 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:12.800225 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.800284 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.800378 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260812 14:46:12.801034 17826 heartbeater.cc:347] Connected to a master server at 127.15.227.254:34591
I20260812 14:46:12.801121 17826 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:12.801261 17826 heartbeater.cc:511] Master 127.15.227.254:34591 requested a full tablet report, sending...
I20260812 14:46:12.801698 17442 ts_manager.cc:194] Registered new tserver with Master: bfd5338515c043a2b53c00a4fe2fa86f (127.15.227.193:37639)
I20260812 14:46:12.802328 17442 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.193:34979
I20260812 14:46:12.805658 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 17567
I20260812 14:46:12.805756 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:46:12.805915 16271 external_mini_cluster.cc:1428] Setting key a4a5df1fe2ed49370db9133a6967407b
I20260812 14:46:12.806715 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34591
--builtin_ntp_servers=127.15.227.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:12.886482 17830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:12.886698 17830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:12.886719 17830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:12.887825 17830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:12.887861 17830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:12.887884 17830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:46:12.889185 17830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:12.889397 17830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:12.889655 17830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:12.894279 17838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:12.894389 17830 server_base.cc:1034] running on GCE node
W20260812 14:46:12.894454 17842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:12.894285 17840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:12.894706 17830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:12.894941 17830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:12.896085 17830 hybrid_clock.cc:648] HybridClock initialized: now 1786545972896068 us; error 28 us; skew 500 ppm
I20260812 14:46:12.897296 17830 webserver.cc:460] Webserver started at http://127.15.227.194:42653/ using document root <none> and password file <none>
I20260812 14:46:12.897529 17830 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:12.897578 17830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:12.897775 17830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:12.898831 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "514c41903e1c4b128a88d9072c10329c"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "4981c85d4412b60b109b2e13bfc0102d"
server_key_iv: "7c738166e8d35769a56e2d1edbafbddf"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.899158 17830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "514c41903e1c4b128a88d9072c10329c"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "4981c85d4412b60b109b2e13bfc0102d"
server_key_iv: "7c738166e8d35769a56e2d1edbafbddf"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.900326 17830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:46:12.900940 17849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.901088 17830 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:12.901167 17830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "514c41903e1c4b128a88d9072c10329c"
format_stamp: "Formatted at 2026-08-12 14:46:12 on dist-test-slave-q9h9"
server_key: "4981c85d4412b60b109b2e13bfc0102d"
server_key_iv: "7c738166e8d35769a56e2d1edbafbddf"
server_key_version: "encryptionkey@0"
I20260812 14:46:12.901234 17830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:12.913570 17830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:12.913802 17830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:12.913915 17830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:12.914152 17830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:12.914440 17830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:12.914474 17830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.914500 17830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:12.914517 17830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:12.919718 17830 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:33361
I20260812 14:46:12.919752 18069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:33361 every 8 connection(s)
I20260812 14:46:12.920015 17830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:46:12.921608 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 17830
I20260812 14:46:12.921694 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:46:12.921849 16271 external_mini_cluster.cc:1428] Setting key 63abe2776e389c213ab1043995ea3a07
I20260812 14:46:12.922910 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34591
--builtin_ntp_servers=127.15.227.212:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:46:12.925526 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.920646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:36699 (local address 127.15.227.254:34591)
0812 14:46:12.920712 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:12.920714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:12.920882 (+ 168us) server_negotiation.cc:407] Connection header received
0812 14:46:12.921290 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:12.921290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:12.921299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:12.921348 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:12.921977 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.922549 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.924014 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.924292 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.924341 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.925242 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:12.925243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:12.925244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:12.925249 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:12.925268 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:12.925273 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:12.925326 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:12.925405 (+ 79us) server_negotiation.cc:299] Negotiation successful
0812 14:46:12.925446 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 14:46:12.925690 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:12.920610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:34591 (local address 127.15.227.194:36699)
0812 14:46:12.920780 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:12.920815 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:12.921215 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:12.921395 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:12.921398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:12.921405 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:12.921898 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.921904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.922657 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.922659 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.923914 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:12.923917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:12.924445 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:12.924446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:12.924594 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:12.925069 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:12.925069 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:12.925105 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:12.925132 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:12.925333 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:12.925338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:12.925340 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:46:12.925397 (+ 57us) client_negotiation.cc:240] Negotiation successful
0812 14:46:12.925492 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260812 14:46:12.926229 18071 heartbeater.cc:347] Connected to a master server at 127.15.227.254:34591
I20260812 14:46:12.926306 18071 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:12.926458 18071 heartbeater.cc:511] Master 127.15.227.254:34591 requested a full tablet report, sending...
I20260812 14:46:12.926813 17442 ts_manager.cc:194] Registered new tserver with Master: 514c41903e1c4b128a88d9072c10329c (127.15.227.194:33361)
I20260812 14:46:12.927258 17442 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.194:36699
W20260812 14:46:13.015005 18082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:13.015192 18082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:13.015213 18082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:13.016269 18082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:13.016300 18082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:13.016324 18082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:46:13.017585 18082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:34591
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:13.017750 18082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:13.017979 18082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:13.022651 18091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:13.022737 18094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:13.022679 18090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:13.022899 18082 server_base.cc:1034] running on GCE node
I20260812 14:46:13.023088 18082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:13.023325 18082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:13.024453 18082 hybrid_clock.cc:648] HybridClock initialized: now 1786545973024443 us; error 26 us; skew 500 ppm
I20260812 14:46:13.025591 18082 webserver.cc:460] Webserver started at http://127.15.227.195:44055/ using document root <none> and password file <none>
I20260812 14:46:13.025791 18082 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:13.025835 18082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:13.025947 18082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:13.026970 18082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "582019f1ac7c48f9a3ebb5da2521ebcd"
format_stamp: "Formatted at 2026-08-12 14:46:13 on dist-test-slave-q9h9"
server_key: "25e44200e1a2d331f8e0ffbc31ebda41"
server_key_iv: "0250275bec10565e74a66ac9073d293a"
server_key_version: "encryptionkey@0"
I20260812 14:46:13.027258 18082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "582019f1ac7c48f9a3ebb5da2521ebcd"
format_stamp: "Formatted at 2026-08-12 14:46:13 on dist-test-slave-q9h9"
server_key: "25e44200e1a2d331f8e0ffbc31ebda41"
server_key_iv: "0250275bec10565e74a66ac9073d293a"
server_key_version: "encryptionkey@0"
I20260812 14:46:13.028363 18082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:46:13.028956 18104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:13.029120 18082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:13.029202 18082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "582019f1ac7c48f9a3ebb5da2521ebcd"
format_stamp: "Formatted at 2026-08-12 14:46:13 on dist-test-slave-q9h9"
server_key: "25e44200e1a2d331f8e0ffbc31ebda41"
server_key_iv: "0250275bec10565e74a66ac9073d293a"
server_key_version: "encryptionkey@0"
I20260812 14:46:13.029268 18082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:13.055667 18082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:13.055940 18082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:13.056066 18082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:13.056273 18082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:13.056594 18082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:13.056631 18082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:13.056658 18082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:13.056684 18082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:13.062004 18082 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:44917
I20260812 14:46:13.062075 18337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:44917 every 8 connection(s)
I20260812 14:46:13.062384 18082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:46:13.067236 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.063061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:60105 (local address 127.15.227.254:34591)
0812 14:46:13.063121 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.063123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.063336 (+ 213us) server_negotiation.cc:407] Connection header received
0812 14:46:13.063839 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.063840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.063854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.063906 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:13.064565 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.065062 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.066031 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.066224 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.066284 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.066983 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:13.066985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:13.066985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:13.066990 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:13.067006 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:13.067010 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:13.067052 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.067124 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.067156 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:46:13.067417 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.063041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:34591 (local address 127.15.227.195:60105)
0812 14:46:13.063215 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:13.063255 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:13.063733 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:13.063974 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:13.063978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:13.063986 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:13.064468 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.064473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.065152 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.065154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.065921 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.065923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.066301 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.066302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.066420 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.066857 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:13.066857 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:13.066884 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:13.066906 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:13.067058 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:13.067063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:13.067066 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:13.067122 (+ 56us) client_negotiation.cc:240] Negotiation successful
0812 14:46:13.067220 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260812 14:46:13.067835 18338 heartbeater.cc:347] Connected to a master server at 127.15.227.254:34591
I20260812 14:46:13.067916 18338 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:13.068082 18338 heartbeater.cc:511] Master 127.15.227.254:34591 requested a full tablet report, sending...
I20260812 14:46:13.068403 17442 ts_manager.cc:194] Registered new tserver with Master: 582019f1ac7c48f9a3ebb5da2521ebcd (127.15.227.195:44917)
I20260812 14:46:13.068820 17442 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.195:60105
I20260812 14:46:13.069509 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18082
I20260812 14:46:13.069608 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:46:13.069820 16271 external_mini_cluster.cc:1428] Setting key 0fce682acb88f91bd2cad5961bc1f06b
I20260812 14:46:13.070574 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:46:13.074381 17525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.071783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.15.227.254:34591)
0812 14:46:13.071837 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.071839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.071952 (+ 113us) server_negotiation.cc:407] Connection header received
0812 14:46:13.072013 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.072014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.072025 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.072069 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:13.072221 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.072673 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.073382 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.073576 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.073635 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.074169 (+ 534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:13.074170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:13.074171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:13.074175 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:13.074188 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:13.074192 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:13.074234 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.074285 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.074307 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:46:13.075623 17442 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:46:13.077100 17442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:46:13.083000 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.078996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:44917 (local address 127.0.0.1:35026)
0812 14:46:13.079348 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:13.079355 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:13.079389 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:13.080022 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:13.080023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:13.080025 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:13.080060 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.080061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.081107 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.081108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.081948 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.081950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.082798 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.082798 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.082855 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.082866 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 14:46:13.082917 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:46:13.082951 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":2,"thread_start_us":98,"threads_started":1}
I20260812 14:46:13.083142 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.079203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35026 (local address 127.15.227.195:44917)
0812 14:46:13.079896 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.079900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.079908 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:13.079937 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.079939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.079952 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.080022 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:13.080131 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.081038 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.082038 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.082717 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.082798 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.082876 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.082941 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.082974 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":73,"threads_started":1}
I20260812 14:46:13.083410 18352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.078984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:33361 (local address 127.0.0.1:44032)
0812 14:46:13.079152 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:13.079192 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:13.079250 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:13.079783 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:13.079784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:13.079786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:13.079820 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.079822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.080763 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.080764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.082074 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.082076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.083202 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.083203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.083269 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.083273 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:13.083327 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:46:13.083366 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":38,"thread_start_us":50,"threads_started":1}
I20260812 14:46:13.083482 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.079133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.15.227.194:33361)
0812 14:46:13.079622 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.079626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.079635 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:46:13.079695 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.079697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.079712 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.079786 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:13.079904 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.080681 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.082152 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.082983 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.083050 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.083134 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.083353 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.083398 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":81,"threads_started":1}
I20260812 14:46:13.083729 18202 tablet_service.cc:1467] Processing CreateTablet for tablet d9bc7ff1d5f74083b75543b7e81f6b0d (DEFAULT_TABLE table=test-table [id=d82be501715b4b25838bce3678d61c71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:13.084039 18202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9bc7ff1d5f74083b75543b7e81f6b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:13.084115 17957 tablet_service.cc:1467] Processing CreateTablet for tablet d9bc7ff1d5f74083b75543b7e81f6b0d (DEFAULT_TABLE table=test-table [id=d82be501715b4b25838bce3678d61c71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:13.084443 17957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9bc7ff1d5f74083b75543b7e81f6b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:13.086622 18358 tablet_bootstrap.cc:492] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Bootstrap starting.
I20260812 14:46:13.087078 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.078996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:37639 (local address 127.0.0.1:55360)
0812 14:46:13.079316 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:13.079326 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:13.079369 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:13.079524 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:13.079527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:13.079528 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:13.079573 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.079574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.080828 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.080829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.082073 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.082076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.086851 (+ 4775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.086852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.086914 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.086918 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:13.086969 (+ 51us) client_negotiation.cc:240] Negotiation successful
0812 14:46:13.087010 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":32,"thread_start_us":105,"threads_started":1}
I20260812 14:46:13.087096 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.079225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.15.227.193:37639)
0812 14:46:13.079394 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.079398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.079407 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:46:13.079438 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.079440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.079451 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.079511 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:13.079649 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.080755 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.083174 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.086780 (+ 3606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.086853 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.086934 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.086985 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.087023 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
I20260812 14:46:13.087141 18360 tablet_bootstrap.cc:492] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Bootstrap starting.
I20260812 14:46:13.087715 18358 tablet_bootstrap.cc:654] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:13.087715 18360 tablet_bootstrap.cc:654] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:13.087670 17695 tablet_service.cc:1467] Processing CreateTablet for tablet d9bc7ff1d5f74083b75543b7e81f6b0d (DEFAULT_TABLE table=test-table [id=d82be501715b4b25838bce3678d61c71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:13.087942 17695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9bc7ff1d5f74083b75543b7e81f6b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:13.088038 18360 log.cc:824] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:13.088038 18358 log.cc:824] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:13.088721 18360 tablet_bootstrap.cc:492] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: No bootstrap required, opened a new log
I20260812 14:46:13.088801 18360 ts_tablet_manager.cc:1397] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:13.088975 18358 tablet_bootstrap.cc:492] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: No bootstrap required, opened a new log
I20260812 14:46:13.089039 18358 ts_tablet_manager.cc:1397] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:13.090710 18366 tablet_bootstrap.cc:492] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f: Bootstrap starting.
I20260812 14:46:13.091334 18366 tablet_bootstrap.cc:654] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:13.091270 18360 raft_consensus.cc:348] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.091431 18360 raft_consensus.cc:374] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:13.091472 18360 raft_consensus.cc:729] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 514c41903e1c4b128a88d9072c10329c, State: Initialized, Role: FOLLOWER
I20260812 14:46:13.091516 18358 raft_consensus.cc:348] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.091585 18360 consensus_queue.cc:260] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.091629 18366 log.cc:824] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:13.091652 18358 raft_consensus.cc:374] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:13.091687 18358 raft_consensus.cc:729] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 582019f1ac7c48f9a3ebb5da2521ebcd, State: Initialized, Role: FOLLOWER
I20260812 14:46:13.091851 18360 ts_tablet_manager.cc:1428] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:46:13.091817 18358 consensus_queue.cc:260] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.091969 18071 heartbeater.cc:503] Master 127.15.227.254:34591 was elected leader, sending a full tablet report...
I20260812 14:46:13.092056 18358 ts_tablet_manager.cc:1428] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260812 14:46:13.092155 18338 heartbeater.cc:503] Master 127.15.227.254:34591 was elected leader, sending a full tablet report...
I20260812 14:46:13.092361 18366 tablet_bootstrap.cc:492] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f: No bootstrap required, opened a new log
I20260812 14:46:13.092424 18366 ts_tablet_manager.cc:1397] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:13.094292 18366 raft_consensus.cc:348] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.094440 18366 raft_consensus.cc:374] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:13.094485 18366 raft_consensus.cc:729] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfd5338515c043a2b53c00a4fe2fa86f, State: Initialized, Role: FOLLOWER
I20260812 14:46:13.094589 18366 consensus_queue.cc:260] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.094834 18366 ts_tablet_manager.cc:1428] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:46:13.094884 17826 heartbeater.cc:503] Master 127.15.227.254:34591 was elected leader, sending a full tablet report...
W20260812 14:46:13.170490 18076 tablet.cc:2369] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:46:13.188040 18373 raft_consensus.cc:482] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:13.188144 18373 raft_consensus.cc:504] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.188503 18373 leader_election.cc:288] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 514c41903e1c4b128a88d9072c10329c (127.15.227.194:33361), 582019f1ac7c48f9a3ebb5da2521ebcd (127.15.227.195:44917)
I20260812 14:46:13.191134 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.188670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:33361 (local address 127.15.227.193:59279)
0812 14:46:13.188724 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:13.188735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:13.188775 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:13.188960 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:13.188962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:13.188964 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:13.189028 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.189030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.189623 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.189624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.190393 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.190396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.190907 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.190908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.190971 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.190975 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:13.191018 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:46:13.191061 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 14:46:13.191135 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.188733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59279 (local address 127.15.227.194:33361)
0812 14:46:13.188845 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.188847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.188855 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:13.188883 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.188884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.188893 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.188945 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:13.189090 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.189557 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.190479 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.190841 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.190903 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.190912 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.191039 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.191078 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 14:46:13.191233 18355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.188745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:50047 (local address 127.15.227.195:44917)
0812 14:46:13.188810 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.188812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.188925 (+ 113us) server_negotiation.cc:407] Connection header received
0812 14:46:13.188972 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.188973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.188982 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.189024 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:13.189185 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.189648 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.190578 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.190925 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.190986 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.190994 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.191128 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.191173 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:46:13.191409 18376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.188672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:44917 (local address 127.15.227.193:50047)
0812 14:46:13.188863 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:13.188871 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:13.188921 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:13.189034 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:13.189035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:13.189037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:13.189108 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.189109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.189721 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.189721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.190495 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:13.190497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.190996 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.190996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:13.191064 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.191067 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:13.191112 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:46:13.191150 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":15,"thread_start_us":110,"threads_started":1}
I20260812 14:46:13.191449 17994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bc7ff1d5f74083b75543b7e81f6b0d" candidate_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "514c41903e1c4b128a88d9072c10329c" is_pre_election: true
I20260812 14:46:13.191597 17994 raft_consensus.cc:2454] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfd5338515c043a2b53c00a4fe2fa86f in term 0.
I20260812 14:46:13.191783 18250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bc7ff1d5f74083b75543b7e81f6b0d" candidate_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" is_pre_election: true
I20260812 14:46:13.191820 17595 leader_election.cc:302] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 514c41903e1c4b128a88d9072c10329c, bfd5338515c043a2b53c00a4fe2fa86f; no voters:
I20260812 14:46:13.191932 18250 raft_consensus.cc:2454] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfd5338515c043a2b53c00a4fe2fa86f in term 0.
I20260812 14:46:13.191952 18373 raft_consensus.cc:2781] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:46:13.192034 18373 raft_consensus.cc:482] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:46:13.192065 18373 raft_consensus.cc:3037] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:13.192768 18373 raft_consensus.cc:504] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.192900 18373 leader_election.cc:288] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [CANDIDATE]: Term 1 election: Requested vote from peers 514c41903e1c4b128a88d9072c10329c (127.15.227.194:33361), 582019f1ac7c48f9a3ebb5da2521ebcd (127.15.227.195:44917)
I20260812 14:46:13.193058 18250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bc7ff1d5f74083b75543b7e81f6b0d" candidate_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd"
I20260812 14:46:13.193076 17994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9bc7ff1d5f74083b75543b7e81f6b0d" candidate_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "514c41903e1c4b128a88d9072c10329c"
I20260812 14:46:13.193132 18250 raft_consensus.cc:3037] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:13.193138 17994 raft_consensus.cc:3037] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:13.193845 17994 raft_consensus.cc:2454] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd5338515c043a2b53c00a4fe2fa86f in term 1.
I20260812 14:46:13.193926 18250 raft_consensus.cc:2454] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfd5338515c043a2b53c00a4fe2fa86f in term 1.
I20260812 14:46:13.194007 17595 leader_election.cc:302] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 514c41903e1c4b128a88d9072c10329c, bfd5338515c043a2b53c00a4fe2fa86f; no voters:
I20260812 14:46:13.194118 18373 raft_consensus.cc:2781] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:13.194269 18373 raft_consensus.cc:686] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [term 1 LEADER]: Becoming Leader. State: Replica: bfd5338515c043a2b53c00a4fe2fa86f, State: Running, Role: LEADER
I20260812 14:46:13.194351 18373 consensus_queue.cc:237] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } }
I20260812 14:46:13.195139 17442 catalog_manager.cc:5495] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f reported cstate change: term changed from 0 to 1, leader changed from <none> to bfd5338515c043a2b53c00a4fe2fa86f (127.15.227.193). New cstate: current_term: 1 leader_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfd5338515c043a2b53c00a4fe2fa86f" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 } health_report { overall_health: UNKNOWN } } }
I20260812 14:46:13.225378 18357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.223360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.15.227.193:37639)
0812 14:46:13.223412 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:13.223414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:13.223421 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:13.223473 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:13.223474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:13.223490 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:13.223546 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:13.223688 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.224113 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.224814 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:13.224982 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:13.225023 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:13.225109 (+ 86us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:13.225220 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:13.225275 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:13.225288 (+ 13us) server_negotiation.cc:299] Negotiation successful
0812 14:46:13.225304 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:46:13.226974 17994 raft_consensus.cc:1261] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 1 FOLLOWER]: Refusing update from remote peer bfd5338515c043a2b53c00a4fe2fa86f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:13.226974 18250 raft_consensus.cc:1261] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 1 FOLLOWER]: Refusing update from remote peer bfd5338515c043a2b53c00a4fe2fa86f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:13.227308 18373 consensus_queue.cc:1035] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "514c41903e1c4b128a88d9072c10329c" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 33361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:13.227444 18379 consensus_queue.cc:1035] T d9bc7ff1d5f74083b75543b7e81f6b0d P bfd5338515c043a2b53c00a4fe2fa86f [LEADER]: Connected to new peer: Peer: permanent_uuid: "582019f1ac7c48f9a3ebb5da2521ebcd" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:13.229696 18390 mvcc.cc:205] Tried to move back new op lower bound from 7317692306335518720 to 7317692306204626944. Current Snapshot: MvccSnapshot[applied={T|T < 7317692306335518720}]
I20260812 14:46:13.230265 18388 mvcc.cc:205] Tried to move back new op lower bound from 7317692306335518720 to 7317692306204626944. Current Snapshot: MvccSnapshot[applied={T|T < 7317692306335518720}]
I20260812 14:46:13.231799 17442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:13.231873 17442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:13.232553 17442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 052f1807bf884238a7c6ec38b3897386: Sending DeleteTablet for 3 replicas of tablet d9bc7ff1d5f74083b75543b7e81f6b0d
I20260812 14:46:13.232882 17957 tablet_service.cc:1514] Processing DeleteTablet for tablet d9bc7ff1d5f74083b75543b7e81f6b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:13 UTC) from {username='slave'} at 127.0.0.1:44032
I20260812 14:46:13.232919 18202 tablet_service.cc:1514] Processing DeleteTablet for tablet d9bc7ff1d5f74083b75543b7e81f6b0d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:13 UTC) from {username='slave'} at 127.0.0.1:35026
I20260812 14:46:13.233134 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 17567
I20260812 14:46:13.233618 18407 tablet_replica.cc:330] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: stopping tablet replica
I20260812 14:46:13.233647 18408 tablet_replica.cc:330] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: stopping tablet replica
I20260812 14:46:13.233732 18408 raft_consensus.cc:2229] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:13.233774 18407 raft_consensus.cc:2229] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:13.233835 18408 raft_consensus.cc:2258] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:13.233855 18407 raft_consensus.cc:2258] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:13.236435 18407 ts_tablet_manager.cc:1905] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:13.238224 18407 ts_tablet_manager.cc:1918] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:13.238305 18407 log.cc:1196] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/d9bc7ff1d5f74083b75543b7e81f6b0d
I20260812 14:46:13.238575 18407 ts_tablet_manager.cc:1939] T d9bc7ff1d5f74083b75543b7e81f6b0d P 514c41903e1c4b128a88d9072c10329c: Deleting consensus metadata
I20260812 14:46:13.239105 17416 catalog_manager.cc:4841] TS 514c41903e1c4b128a88d9072c10329c (127.15.227.194:33361): tablet d9bc7ff1d5f74083b75543b7e81f6b0d (table test-table [id=d82be501715b4b25838bce3678d61c71]) successfully deleted
I20260812 14:46:13.244427 18408 ts_tablet_manager.cc:1905] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:13.245561 18408 ts_tablet_manager.cc:1918] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:13.245625 18408 log.cc:1196] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/d9bc7ff1d5f74083b75543b7e81f6b0d
I20260812 14:46:13.245819 18408 ts_tablet_manager.cc:1939] T d9bc7ff1d5f74083b75543b7e81f6b0d P 582019f1ac7c48f9a3ebb5da2521ebcd: Deleting consensus metadata
I20260812 14:46:13.246219 17421 catalog_manager.cc:4841] TS 582019f1ac7c48f9a3ebb5da2521ebcd (127.15.227.195:44917): tablet d9bc7ff1d5f74083b75543b7e81f6b0d (table test-table [id=d82be501715b4b25838bce3678d61c71]) successfully deleted
W20260812 14:46:13.250084 17418 connection.cc:535] client connection to 127.15.227.193:37639 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260812 14:46:13.250185 17418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 14:46:13.250381 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 17830
I20260812 14:46:13.252236 18351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:13.250488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:37639 (local address 127.0.0.1:55386)
0812 14:46:13.252144 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:13.252198 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1629}
W20260812 14:46:13.252359 17418 catalog_manager.cc:4584] TS bfd5338515c043a2b53c00a4fe2fa86f (127.15.227.193:37639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9bc7ff1d5f74083b75543b7e81f6b0d: Network error: Client connection negotiation failed: client connection to 127.15.227.193:37639: connect: Connection refused (error 111)
I20260812 14:46:13.265343 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18082
I20260812 14:46:13.276929 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 17383
2026-08-12T14:46:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (798 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:13 dist-test-slave-q9h9 krb5kdc[18414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:13 dist-test-slave-q9h9 krb5kdc[18414](info): set up 2 sockets
Aug 12 14:46:13 dist-test-slave-q9h9 krb5kdc[18414](info): commencing operation
krb5kdc: starting...
W20260812 14:46:14.468397 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:14 dist-test-slave-q9h9 krb5kdc[18414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:14.574115 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:35099
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:35099
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:14.652225 18444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:14.652405 18444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:14.652436 18444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:14.653525 18444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:14.653563 18444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:14.653581 18444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:14.653597 18444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:14.653611 18444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:14.655059 18444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:35099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:35099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:14.655292 18444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:14.655484 18444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:14.659938 18457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:14.659953 18461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:14.660032 18444 server_base.cc:1034] running on GCE node
W20260812 14:46:14.659951 18459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:14.660354 18444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:14.660624 18444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:14.661778 18444 hybrid_clock.cc:648] HybridClock initialized: now 1786545974661766 us; error 30 us; skew 500 ppm
Aug 12 14:46:14 dist-test-slave-q9h9 krb5kdc[18414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:14.664111 18444 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:14.664534 18444 webserver.cc:460] Webserver started at http://127.15.227.254:40347/ using document root <none> and password file <none>
I20260812 14:46:14.664746 18444 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:14.664793 18444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:14.664911 18444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:14.665859 18444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "82dcded8ae934a8e98f3bb8164db356f"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "db886373157ee782523f4f55f3f0ec62"
server_key_iv: "851f29294390d7115ad0199e2ffef302"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.666194 18444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "82dcded8ae934a8e98f3bb8164db356f"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "db886373157ee782523f4f55f3f0ec62"
server_key_iv: "851f29294390d7115ad0199e2ffef302"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.667392 18444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:46:14.668053 18474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.668196 18444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:14.668283 18444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "82dcded8ae934a8e98f3bb8164db356f"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "db886373157ee782523f4f55f3f0ec62"
server_key_iv: "851f29294390d7115ad0199e2ffef302"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.668344 18444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:14.679981 18444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:14.681650 18444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:14.681774 18444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:14.685601 18444 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:35099
I20260812 14:46:14.685639 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:35099 every 8 connection(s)
I20260812 14:46:14.685937 18444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:14.686741 18551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:14.688021 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18444
I20260812 14:46:14.688155 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:14.688372 16271 external_mini_cluster.cc:1428] Setting key f1a249593f54cda87815657fd9dac648
I20260812 14:46:14.689206 18551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f: Bootstrap starting.
I20260812 14:46:14.689797 18551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:14.690176 18551 log.cc:824] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:14.690871 18551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f: No bootstrap required, opened a new log
I20260812 14:46:14.692577 18551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82dcded8ae934a8e98f3bb8164db356f" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35099 } }
I20260812 14:46:14.692703 18551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:14.692749 18551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82dcded8ae934a8e98f3bb8164db356f, State: Initialized, Role: FOLLOWER
I20260812 14:46:14.692867 18551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82dcded8ae934a8e98f3bb8164db356f" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35099 } }
I20260812 14:46:14.692942 18551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:14.692973 18551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:14.693017 18551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:14.693521 18551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82dcded8ae934a8e98f3bb8164db356f" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35099 } }
I20260812 14:46:14.693606 18551 leader_election.cc:302] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82dcded8ae934a8e98f3bb8164db356f; no voters:
I20260812 14:46:14.693758 18551 leader_election.cc:288] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:14.693805 18562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:14.693953 18562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [term 1 LEADER]: Becoming Leader. State: Replica: 82dcded8ae934a8e98f3bb8164db356f, State: Running, Role: LEADER
I20260812 14:46:14.693998 18551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:14.694059 18562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82dcded8ae934a8e98f3bb8164db356f" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35099 } }
I20260812 14:46:14.694355 18562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82dcded8ae934a8e98f3bb8164db356f. Latest consensus state: current_term: 1 leader_uuid: "82dcded8ae934a8e98f3bb8164db356f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82dcded8ae934a8e98f3bb8164db356f" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35099 } } }
I20260812 14:46:14.694365 18564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82dcded8ae934a8e98f3bb8164db356f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82dcded8ae934a8e98f3bb8164db356f" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35099 } } }
I20260812 14:46:14.694439 18562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:14.694443 18564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f [sys.catalog]: This master's current role is: LEADER
Aug 12 14:46:14 dist-test-slave-q9h9 krb5kdc[18414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:14.694648 18572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:14.695178 18572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:14.697168 18572 catalog_manager.cc:1347] Generated new cluster ID: 0266255fa7404680a714e07df303b495
I20260812 14:46:14.697220 18572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:14.698621 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:14.688879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.15.227.254:35099)
0812 14:46:14.689035 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:14.689044 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:14.689068 (+ 24us) server_negotiation.cc:407] Connection header received
0812 14:46:14.689474 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:14.689481 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:14.689533 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:14.689642 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:14.690439 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:14.691352 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:14.692137 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:14.692525 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:14.692637 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:14.695114 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:14.695117 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:14.695118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:14.695137 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:14.696928 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:14.697290 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:14.697293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:14.697293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:14.697391 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:14.697620 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:14.697620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:14.697621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:14.697894 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:14.698044 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:14.698301 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 14:46:14.698426 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260812 14:46:14.705652 18572 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:14.706270 18572 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:14.708343 18572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f: Generated new TSK 0
I20260812 14:46:14.708475 18572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:46:14.738806 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35099
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:14.817698 18593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:14.817876 18593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:14.817907 18593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:14.819041 18593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:14.819084 18593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:14.819119 18593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:46:14.820537 18593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:14.820755 18593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:14.820961 18593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:14.825531 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:14.825657 18603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:14.825683 18593 server_base.cc:1034] running on GCE node
W20260812 14:46:14.825546 18601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:14.825920 18593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:14.826159 18593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:14.827286 18593 hybrid_clock.cc:648] HybridClock initialized: now 1786545974827276 us; error 28 us; skew 500 ppm
Aug 12 14:46:14 dist-test-slave-q9h9 krb5kdc[18414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:14.829569 18593 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:46:14.829978 18593 webserver.cc:460] Webserver started at http://127.15.227.193:38609/ using document root <none> and password file <none>
I20260812 14:46:14.830250 18593 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:14.830298 18593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:14.830410 18593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:14.831349 18593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "fb199d5f499b40e081eec6b39bb23637"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "154851d32f3ec7f0e41a767d8d670935"
server_key_iv: "84900d01f8978fd33d928d42482f42b2"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.831652 18593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "fb199d5f499b40e081eec6b39bb23637"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "154851d32f3ec7f0e41a767d8d670935"
server_key_iv: "84900d01f8978fd33d928d42482f42b2"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.832849 18593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:46:14.833524 18618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.833662 18593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:14.833738 18593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "fb199d5f499b40e081eec6b39bb23637"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "154851d32f3ec7f0e41a767d8d670935"
server_key_iv: "84900d01f8978fd33d928d42482f42b2"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.833801 18593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:14.843860 18593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:14.845757 18593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:14.845880 18593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:14.846108 18593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:14.846387 18593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:14.846419 18593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.846447 18593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:14.846470 18593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.851923 18593 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:39723
I20260812 14:46:14.851979 18852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:39723 every 8 connection(s)
I20260812 14:46:14.852241 18593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:46:14.853206 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18593
I20260812 14:46:14.853322 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:46:14.853518 16271 external_mini_cluster.cc:1428] Setting key 3f627bf90514eddace305c57a74d231f
Aug 12 14:46:14 dist-test-slave-q9h9 krb5kdc[18414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:14.861336 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:14.852867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:58727 (local address 127.15.227.254:35099)
0812 14:46:14.852916 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:14.852918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:14.853097 (+ 179us) server_negotiation.cc:407] Connection header received
0812 14:46:14.853597 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:14.853598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:14.853612 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:14.853686 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:14.854687 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:14.855354 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:14.856333 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:14.856659 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:14.856729 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:14.858797 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:14.858799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:14.858799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:14.858807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:14.860047 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:14.860500 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:14.860500 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:14.860501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:14.860545 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:14.860784 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:14.860785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:14.860785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:14.860875 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:14.860918 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:14.861210 (+ 292us) server_negotiation.cc:299] Negotiation successful
0812 14:46:14.861275 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:46:14.861617 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:14.852834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:35099 (local address 127.15.227.193:58727)
0812 14:46:14.852990 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:14.853022 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:14.853499 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:14.853761 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:14.853765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:14.854417 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:14.854588 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:14.854593 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:14.855445 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:14.855446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:14.856249 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:14.856252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:14.856751 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:14.856752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:14.856847 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:14.857244 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:14.857252 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:14.858703 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:14.860121 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:14.860130 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:14.860132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:14.860432 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:14.860597 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:14.860597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:14.860598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:14.860727 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:14.860927 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:14.860932 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:14.861082 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 14:46:14.861199 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 14:46:14.861358 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260812 14:46:14.862159 18854 heartbeater.cc:347] Connected to a master server at 127.15.227.254:35099
I20260812 14:46:14.862246 18854 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:14.862413 18854 heartbeater.cc:511] Master 127.15.227.254:35099 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
I20260812 14:46:14.862877 18496 ts_manager.cc:194] Registered new tserver with Master: fb199d5f499b40e081eec6b39bb23637 (127.15.227.193:39723)
I20260812 14:46:14.863438 18496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:58727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:46:14.878363 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35099
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:14.957892 18859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:14.958122 18859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:14.958150 18859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:14.959271 18859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:14.959312 18859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:14.959336 18859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:46:14.960747 18859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:14.960944 18859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:14.961156 18859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:14.965643 18878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:14.965698 18871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:14.965680 18872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:14.965824 18859 server_base.cc:1034] running on GCE node
I20260812 14:46:14.966002 18859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:14.966274 18859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:14.967422 18859 hybrid_clock.cc:648] HybridClock initialized: now 1786545974967407 us; error 28 us; skew 500 ppm
Aug 12 14:46:14 dist-test-slave-q9h9 krb5kdc[18414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:14.969748 18859 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:46:14.970193 18859 webserver.cc:460] Webserver started at http://127.15.227.194:46859/ using document root <none> and password file <none>
I20260812 14:46:14.970470 18859 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:14.970520 18859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:14.970645 18859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:14.971696 18859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "f40037c24ca5469496efa6181f9d56df"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "167e4baa3f4ae9ee03a0b0c8649a694f"
server_key_iv: "ebe57733229b499b3a4fddb3bfbf8d51"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.972025 18859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "f40037c24ca5469496efa6181f9d56df"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "167e4baa3f4ae9ee03a0b0c8649a694f"
server_key_iv: "ebe57733229b499b3a4fddb3bfbf8d51"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.973228 18859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:46:14.973842 18888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.973991 18859 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.974094 18859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "f40037c24ca5469496efa6181f9d56df"
format_stamp: "Formatted at 2026-08-12 14:46:14 on dist-test-slave-q9h9"
server_key: "167e4baa3f4ae9ee03a0b0c8649a694f"
server_key_iv: "ebe57733229b499b3a4fddb3bfbf8d51"
server_key_version: "encryptionkey@0"
I20260812 14:46:14.974156 18859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:14.991509 18859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:14.993420 18859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:14.993546 18859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:14.993738 18859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:14.994053 18859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:14.994097 18859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.994128 18859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:14.994151 18859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:14.999617 18859 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:45179
I20260812 14:46:14.999712 19104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:45179 every 8 connection(s)
I20260812 14:46:14.999944 18859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:46:15.002511 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 18859
I20260812 14:46:15.002604 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:46:15.002795 16271 external_mini_cluster.cc:1428] Setting key 3c5461801560c3c4298a9ae24eb04365
Aug 12 14:46:15 dist-test-slave-q9h9 krb5kdc[18414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545974, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:15.009047 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.000613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:43767 (local address 127.15.227.254:35099)
0812 14:46:15.000678 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.000680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.000834 (+ 154us) server_negotiation.cc:407] Connection header received
0812 14:46:15.001236 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.001237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.001248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.001310 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.002172 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.002731 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.003848 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.004037 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.004089 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.006459 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.006461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.006461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:15.006468 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.007745 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:15.008181 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:15.008181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:15.008182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:15.008237 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:15.008482 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:15.008482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:15.008483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:15.008574 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.008615 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.008919 (+ 304us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.008984 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:46:15.009260 19110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.000582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:35099 (local address 127.15.227.194:43767)
0812 14:46:15.000731 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.000763 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.001152 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.001356 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.001358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.001886 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:15.002082 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.002086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.002814 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.002815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.003758 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.003761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.004111 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.004112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.004251 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.004658 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:15.004667 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:15.006375 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:15.007828 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:15.007835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:15.007837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:15.008114 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:15.008290 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:15.008291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:15.008292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:15.008424 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:15.008625 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:15.008631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:15.008790 (+ 159us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.008901 (+ 111us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.009054 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
I20260812 14:46:15.009711 19106 heartbeater.cc:347] Connected to a master server at 127.15.227.254:35099
I20260812 14:46:15.009801 19106 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:15.009959 19106 heartbeater.cc:511] Master 127.15.227.254:35099 requested a full tablet report, sending...
I20260812 14:46:15.010337 18496 ts_manager.cc:194] Registered new tserver with Master: f40037c24ca5469496efa6181f9d56df (127.15.227.194:45179)
I20260812 14:46:15.010826 18496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:43767
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:46:15.026401 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35099
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:15.106050 19115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:15.106256 19115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:15.106278 19115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:15.107352 19115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:15.107383 19115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:15.107406 19115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:46:15.108759 19115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:15.108935 19115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:15.109099 19115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:46:15.113708 19115 server_base.cc:1034] running on GCE node
W20260812 14:46:15.113667 19123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.113667 19130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.113674 19124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:15.114079 19115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:15.114297 19115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:15.115440 19115 hybrid_clock.cc:648] HybridClock initialized: now 1786545975115427 us; error 25 us; skew 500 ppm
Aug 12 14:46:15 dist-test-slave-q9h9 krb5kdc[18414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:15.117686 19115 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:46:15.118105 19115 webserver.cc:460] Webserver started at http://127.15.227.195:36327/ using document root <none> and password file <none>
I20260812 14:46:15.118310 19115 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:15.118358 19115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:15.118474 19115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:15.119381 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "a55ef46fdca64d8ca27eca6512d3c15f"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "42c17fe2a81411f00698c004715f6a99"
server_key_iv: "6848e7c4cfcce65acf1af4a21ab1724a"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.119689 19115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "a55ef46fdca64d8ca27eca6512d3c15f"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "42c17fe2a81411f00698c004715f6a99"
server_key_iv: "6848e7c4cfcce65acf1af4a21ab1724a"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.120915 19115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:46:15.121575 19147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.121726 19115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:15.121806 19115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "a55ef46fdca64d8ca27eca6512d3c15f"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "42c17fe2a81411f00698c004715f6a99"
server_key_iv: "6848e7c4cfcce65acf1af4a21ab1724a"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.121863 19115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:15.161757 19115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:15.163549 19115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:15.163676 19115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:15.163867 19115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:15.164139 19115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:15.164172 19115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.164201 19115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:15.164227 19115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.169740 19115 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:40959
I20260812 14:46:15.169777 19357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:40959 every 8 connection(s)
I20260812 14:46:15.170080 19115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:46:15.170830 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 19115
I20260812 14:46:15.170945 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:46:15.171120 16271 external_mini_cluster.cc:1428] Setting key 68eb55c8823e3bda2cb2ea2e5b7540b3
Aug 12 14:46:15 dist-test-slave-q9h9 krb5kdc[18414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545975, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:15.178292 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.170735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:60943 (local address 127.15.227.254:35099)
0812 14:46:15.170786 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.170789 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.170979 (+ 190us) server_negotiation.cc:407] Connection header received
0812 14:46:15.171478 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.171479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.171493 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.171572 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.172597 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.173084 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.173972 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.174221 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.174268 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.176032 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.176034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.176034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:15.176040 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.177147 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:15.177476 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:15.177477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:15.177477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:15.177522 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:15.177752 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:15.177754 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:15.177754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:15.177869 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.177924 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.178166 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.178231 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:46:15.178444 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.170708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:35099 (local address 127.15.227.195:60943)
0812 14:46:15.170851 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.170888 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.171382 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.171699 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.171704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.172339 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:15.172507 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.172511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.173164 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.173165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.173909 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.173912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.174281 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.174281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.174376 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.174730 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:15.174738 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:15.175954 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:15.177210 (+ 1256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:15.177224 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:15.177225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:15.177422 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:15.177593 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:15.177594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:15.177595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:15.177693 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:15.177921 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:15.177923 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:15.178048 (+ 125us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.178150 (+ 102us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.178264 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260812 14:46:15.178825 19359 heartbeater.cc:347] Connected to a master server at 127.15.227.254:35099
I20260812 14:46:15.178897 19359 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:15.179041 19359 heartbeater.cc:511] Master 127.15.227.254:35099 requested a full tablet report, sending...
I20260812 14:46:15.179368 18496 ts_manager.cc:194] Registered new tserver with Master: a55ef46fdca64d8ca27eca6512d3c15f (127.15.227.195:40959)
I20260812 14:46:15.179726 18496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:60943
I20260812 14:46:15.182669 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:46:15.188912 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.183861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.15.227.254:35099)
0812 14:46:15.183908 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.183910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.184034 (+ 124us) server_negotiation.cc:407] Connection header received
0812 14:46:15.184068 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.184069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.184076 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.184118 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.184591 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.185021 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.185672 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.185838 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.185876 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.186974 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.186975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.186976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:15.186981 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.188036 (+ 1055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:15.188371 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:15.188371 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:15.188372 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:15.188414 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:15.188544 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:15.188544 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:15.188544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:15.188646 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.188736 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.188801 (+ 65us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.188845 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:46:15.190197 18496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:46:15.191637 18496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:46:15.197552 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.193608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:39723 (local address 127.0.0.1:60366)
0812 14:46:15.193784 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.193792 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.193828 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.194151 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.194152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.194154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.194184 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.194185 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.195282 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.195283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.196714 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.196716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.197364 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.197365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.197418 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.197422 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.197448 (+ 26us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.197478 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":44,"threads_started":1}
I20260812 14:46:15.197691 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.193461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.15.227.195:40959)
0812 14:46:15.193621 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.193626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.193843 (+ 217us) server_negotiation.cc:407] Connection header received
0812 14:46:15.193880 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.193882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.193897 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.193976 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.194303 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.195451 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.196642 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.197299 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.197358 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.197403 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.197506 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.197538 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260812 14:46:15.197564 19372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.193425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:45179 (local address 127.0.0.1:43196)
0812 14:46:15.193527 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.193564 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.193628 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.194164 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.194166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.194168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.194212 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.194222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.195626 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.195627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.196371 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.196373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.197244 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.197245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.197326 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.197342 (+ 16us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.197400 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.197455 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":2,"thread_start_us":51,"threads_started":1}
I20260812 14:46:15.197714 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.193698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.15.227.194:45179)
0812 14:46:15.193993 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.193997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.194006 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:46:15.194068 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.194070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.194084 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.194155 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.194371 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.195551 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.196451 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.197160 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.197237 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.197291 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.197597 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.197639 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":84,"threads_started":1}
I20260812 14:46:15.197940 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.193801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60366 (local address 127.15.227.193:39723)
0812 14:46:15.193936 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.193940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.193947 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:15.193990 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.193992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.194007 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.194269 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.194339 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.195189 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.196778 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.197268 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.197743 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.197806 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.197831 (+ 25us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.197870 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260812 14:46:15.198055 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.193429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:40959 (local address 127.0.0.1:41802)
0812 14:46:15.193684 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.193695 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.193744 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.193998 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.194002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.194003 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.194091 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.194093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.195546 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.195548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.196580 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.196583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.197360 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.197361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.197444 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.197448 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.197495 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.197534 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":14,"thread_start_us":48,"threads_started":1}
I20260812 14:46:15.198426 18990 tablet_service.cc:1467] Processing CreateTablet for tablet 6deba5604d6b42e7964079cea47da88e (DEFAULT_TABLE table=test-table [id=82ea3810307d410facd83ff70afbecd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:15.198549 18725 tablet_service.cc:1467] Processing CreateTablet for tablet 6deba5604d6b42e7964079cea47da88e (DEFAULT_TABLE table=test-table [id=82ea3810307d410facd83ff70afbecd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:15.198679 19238 tablet_service.cc:1467] Processing CreateTablet for tablet 6deba5604d6b42e7964079cea47da88e (DEFAULT_TABLE table=test-table [id=82ea3810307d410facd83ff70afbecd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:15.198787 18990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6deba5604d6b42e7964079cea47da88e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:15.198954 19238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6deba5604d6b42e7964079cea47da88e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:15.198985 18725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6deba5604d6b42e7964079cea47da88e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:15.201180 19381 tablet_bootstrap.cc:492] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Bootstrap starting.
I20260812 14:46:15.201784 19381 tablet_bootstrap.cc:654] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:15.202275 19385 tablet_bootstrap.cc:492] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Bootstrap starting.
I20260812 14:46:15.202390 19381 log.cc:824] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:15.202396 19386 tablet_bootstrap.cc:492] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637: Bootstrap starting.
I20260812 14:46:15.202975 19386 tablet_bootstrap.cc:654] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:15.202973 19385 tablet_bootstrap.cc:654] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:15.203187 19381 tablet_bootstrap.cc:492] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: No bootstrap required, opened a new log
I20260812 14:46:15.203258 19381 ts_tablet_manager.cc:1397] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:46:15.203378 19386 log.cc:824] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:15.203807 19385 log.cc:824] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:15.204079 19386 tablet_bootstrap.cc:492] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637: No bootstrap required, opened a new log
I20260812 14:46:15.204147 19386 ts_tablet_manager.cc:1397] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:15.204963 19381 raft_consensus.cc:348] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.205097 19381 raft_consensus.cc:374] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:15.205129 19381 raft_consensus.cc:729] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f40037c24ca5469496efa6181f9d56df, State: Initialized, Role: FOLLOWER
I20260812 14:46:15.205245 19381 consensus_queue.cc:260] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.205499 19381 ts_tablet_manager.cc:1428] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:46:15.205556 19106 heartbeater.cc:503] Master 127.15.227.254:35099 was elected leader, sending a full tablet report...
I20260812 14:46:15.205562 19386 raft_consensus.cc:348] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.205693 19386 raft_consensus.cc:374] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:15.205711 19386 raft_consensus.cc:729] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb199d5f499b40e081eec6b39bb23637, State: Initialized, Role: FOLLOWER
I20260812 14:46:15.205788 19386 consensus_queue.cc:260] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.206071 18854 heartbeater.cc:503] Master 127.15.227.254:35099 was elected leader, sending a full tablet report...
I20260812 14:46:15.206046 19386 ts_tablet_manager.cc:1428] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:46:15.206406 19385 tablet_bootstrap.cc:492] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: No bootstrap required, opened a new log
I20260812 14:46:15.206508 19385 ts_tablet_manager.cc:1397] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260812 14:46:15.207882 19385 raft_consensus.cc:348] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.208006 19385 raft_consensus.cc:374] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:15.208027 19385 raft_consensus.cc:729] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a55ef46fdca64d8ca27eca6512d3c15f, State: Initialized, Role: FOLLOWER
I20260812 14:46:15.208124 19385 consensus_queue.cc:260] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.208364 19385 ts_tablet_manager.cc:1428] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:46:15.208398 19359 heartbeater.cc:503] Master 127.15.227.254:35099 was elected leader, sending a full tablet report...
I20260812 14:46:15.215051 19394 raft_consensus.cc:482] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:15.215128 19394 raft_consensus.cc:504] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.215394 19394 leader_election.cc:288] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f40037c24ca5469496efa6181f9d56df (127.15.227.194:45179), a55ef46fdca64d8ca27eca6512d3c15f (127.15.227.195:40959)
I20260812 14:46:15.217895 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.215581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:48053 (local address 127.15.227.194:45179)
0812 14:46:15.215620 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.215622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.215630 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:15.215700 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.215701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.215710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.215752 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.215865 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.216283 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.217273 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.217613 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.217672 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.217679 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.217811 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.217838 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:46:15.217900 18856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.215528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:45179 (local address 127.15.227.193:48053)
0812 14:46:15.215581 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.215589 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.215626 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.215761 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.215761 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.215762 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.215813 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.215815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.216346 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.216347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.217205 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.217207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.217679 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.217680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.217755 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.217758 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.217801 (+ 43us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.217836 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":23}
I20260812 14:46:15.218051 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.215637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:55477 (local address 127.15.227.195:40959)
0812 14:46:15.215671 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.215673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.215761 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:46:15.215811 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.215812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.215820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.215862 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.215977 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.216402 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.217390 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.217743 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.217802 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.217808 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.217939 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.217969 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:15.218065 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.215533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:40959 (local address 127.15.227.193:55477)
0812 14:46:15.215685 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.215693 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.215758 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.215864 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.215865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.215866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.215925 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.215927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.216467 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.216468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.217321 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.217323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.217811 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.217812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.217882 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.217886 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.217928 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.217963 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":10,"thread_start_us":76,"threads_started":1}
I20260812 14:46:15.218263 19029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deba5604d6b42e7964079cea47da88e" candidate_uuid: "fb199d5f499b40e081eec6b39bb23637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40037c24ca5469496efa6181f9d56df" is_pre_election: true
I20260812 14:46:15.218284 19279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deba5604d6b42e7964079cea47da88e" candidate_uuid: "fb199d5f499b40e081eec6b39bb23637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" is_pre_election: true
I20260812 14:46:15.218396 19279 raft_consensus.cc:2454] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb199d5f499b40e081eec6b39bb23637 in term 0.
I20260812 14:46:15.218397 19029 raft_consensus.cc:2454] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb199d5f499b40e081eec6b39bb23637 in term 0.
I20260812 14:46:15.218601 18621 leader_election.cc:302] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55ef46fdca64d8ca27eca6512d3c15f, fb199d5f499b40e081eec6b39bb23637; no voters:
I20260812 14:46:15.218737 19394 raft_consensus.cc:2781] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:46:15.218794 19394 raft_consensus.cc:482] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:46:15.218813 19394 raft_consensus.cc:3037] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:15.219321 19394 raft_consensus.cc:504] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.219417 19394 leader_election.cc:288] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [CANDIDATE]: Term 1 election: Requested vote from peers f40037c24ca5469496efa6181f9d56df (127.15.227.194:45179), a55ef46fdca64d8ca27eca6512d3c15f (127.15.227.195:40959)
I20260812 14:46:15.219552 19029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deba5604d6b42e7964079cea47da88e" candidate_uuid: "fb199d5f499b40e081eec6b39bb23637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f40037c24ca5469496efa6181f9d56df"
I20260812 14:46:15.219591 19279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6deba5604d6b42e7964079cea47da88e" candidate_uuid: "fb199d5f499b40e081eec6b39bb23637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a55ef46fdca64d8ca27eca6512d3c15f"
I20260812 14:46:15.219635 19029 raft_consensus.cc:3037] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:15.219645 19279 raft_consensus.cc:3037] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:15.220376 19029 raft_consensus.cc:2454] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb199d5f499b40e081eec6b39bb23637 in term 1.
I20260812 14:46:15.220376 19279 raft_consensus.cc:2454] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb199d5f499b40e081eec6b39bb23637 in term 1.
I20260812 14:46:15.220525 18621 leader_election.cc:302] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a55ef46fdca64d8ca27eca6512d3c15f, fb199d5f499b40e081eec6b39bb23637; no voters:
I20260812 14:46:15.220654 19394 raft_consensus.cc:2781] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:15.220790 19394 raft_consensus.cc:686] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [term 1 LEADER]: Becoming Leader. State: Replica: fb199d5f499b40e081eec6b39bb23637, State: Running, Role: LEADER
I20260812 14:46:15.220865 19394 consensus_queue.cc:237] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } }
I20260812 14:46:15.221508 18495 catalog_manager.cc:5495] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb199d5f499b40e081eec6b39bb23637 (127.15.227.193). New cstate: current_term: 1 leader_uuid: "fb199d5f499b40e081eec6b39bb23637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb199d5f499b40e081eec6b39bb23637" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 } health_report { overall_health: UNKNOWN } } }
I20260812 14:46:15.226738 19379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.224859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.15.227.193:39723)
0812 14:46:15.224900 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.224901 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.224910 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:46:15.224935 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.224936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.224949 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.225001 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:15.225085 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.225516 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.226207 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.226374 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.226424 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.226536 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:15.226608 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:15.226668 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.226689 (+ 21us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.226697 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260812 14:46:15.228111 19279 raft_consensus.cc:1261] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 1 FOLLOWER]: Refusing update from remote peer fb199d5f499b40e081eec6b39bb23637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:15.228142 19029 raft_consensus.cc:1261] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 1 FOLLOWER]: Refusing update from remote peer fb199d5f499b40e081eec6b39bb23637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:15.228384 19394 consensus_queue.cc:1035] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f40037c24ca5469496efa6181f9d56df" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 45179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:15.228497 19400 consensus_queue.cc:1035] T 6deba5604d6b42e7964079cea47da88e P fb199d5f499b40e081eec6b39bb23637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a55ef46fdca64d8ca27eca6512d3c15f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 40959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:15.230818 19413 mvcc.cc:205] Tried to move back new op lower bound from 7317692314532339712 to 7317692314505191424. Current Snapshot: MvccSnapshot[applied={T|T < 7317692314532339712}]
I20260812 14:46:15.232736 18496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:15.232811 18496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:15.233384 18496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82dcded8ae934a8e98f3bb8164db356f: Sending DeleteTablet for 3 replicas of tablet 6deba5604d6b42e7964079cea47da88e
I20260812 14:46:15.233664 19238 tablet_service.cc:1514] Processing DeleteTablet for tablet 6deba5604d6b42e7964079cea47da88e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:15 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:41802
I20260812 14:46:15.233662 18990 tablet_service.cc:1514] Processing DeleteTablet for tablet 6deba5604d6b42e7964079cea47da88e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:15 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:43196
I20260812 14:46:15.233702 18725 tablet_service.cc:1514] Processing DeleteTablet for tablet 6deba5604d6b42e7964079cea47da88e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:15 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:60366
I20260812 14:46:15.233887 19433 tablet_replica.cc:330] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: stopping tablet replica
I20260812 14:46:15.234036 19430 tablet_replica.cc:330] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: stopping tablet replica
I20260812 14:46:15.234141 19430 raft_consensus.cc:2229] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:15.234203 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18593
I20260812 14:46:15.234238 19430 raft_consensus.cc:2258] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:15.240217 19430 ts_tablet_manager.cc:1905] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:15.241998 19430 ts_tablet_manager.cc:1918] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:15.242164 19430 log.cc:1196] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/6deba5604d6b42e7964079cea47da88e
I20260812 14:46:15.242480 19430 ts_tablet_manager.cc:1939] T 6deba5604d6b42e7964079cea47da88e P f40037c24ca5469496efa6181f9d56df: Deleting consensus metadata
I20260812 14:46:15.251279 18477 catalog_manager.cc:4841] TS f40037c24ca5469496efa6181f9d56df (127.15.227.194:45179): tablet 6deba5604d6b42e7964079cea47da88e (table test-table [id=82ea3810307d410facd83ff70afbecd7]) successfully deleted
I20260812 14:46:15.252543 19433 raft_consensus.cc:2229] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:15.252657 19433 raft_consensus.cc:2258] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:15.253145 19433 ts_tablet_manager.cc:1905] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:15.254294 19433 ts_tablet_manager.cc:1918] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:15.254359 19433 log.cc:1196] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/6deba5604d6b42e7964079cea47da88e
I20260812 14:46:15.254575 19433 ts_tablet_manager.cc:1939] T 6deba5604d6b42e7964079cea47da88e P a55ef46fdca64d8ca27eca6512d3c15f: Deleting consensus metadata
I20260812 14:46:15.254921 18476 catalog_manager.cc:4841] TS a55ef46fdca64d8ca27eca6512d3c15f (127.15.227.195:40959): tablet 6deba5604d6b42e7964079cea47da88e (table test-table [id=82ea3810307d410facd83ff70afbecd7]) successfully deleted
W20260812 14:46:15.256685 18477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:39723 (error 108)
I20260812 14:46:15.257093 19373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.256983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:39723 (local address 127.0.0.1:60374)
0812 14:46:15.257026 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.257068 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:46:15.257205 18477 catalog_manager.cc:4584] TS fb199d5f499b40e081eec6b39bb23637 (127.15.227.193:39723): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6deba5604d6b42e7964079cea47da88e: Network error: Client connection negotiation failed: client connection to 127.15.227.193:39723: connect: Connection refused (error 111)
I20260812 14:46:15.257227 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18859
I20260812 14:46:15.265044 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 19115
I20260812 14:46:15.294064 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 18444
2026-08-12T14:46:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2015 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T14:46:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:15Z Disabled control of system clock
I20260812 14:46:15.325047 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:35921
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:38525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:35921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:15.402952 19440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:15.403131 19440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:15.403151 19440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:15.404260 19440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:15.404299 19440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:15.404311 19440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:15.404335 19440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:15.404359 19440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:15.405656 19440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:35921
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:35921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:15.405865 19440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:15.406136 19440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:15.410437 19455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:15.410557 19440 server_base.cc:1034] running on GCE node
W20260812 14:46:15.410436 19457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.410509 19452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:15.410897 19440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:15.411139 19440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:15.412271 19440 hybrid_clock.cc:648] HybridClock initialized: now 1786545975412258 us; error 29 us; skew 500 ppm
I20260812 14:46:15.416615 19440 webserver.cc:460] Webserver started at http://127.15.227.254:33065/ using document root <none> and password file <none>
I20260812 14:46:15.416822 19440 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:15.416865 19440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:15.416985 19440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:15.417959 19440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "fa4d8ccd744d456daab01f766fa8984a"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "c8bebabbfb9a229646f306c3cc1787fc"
server_key_iv: "2e70d5926c3abe85fd693fb3ed25838a"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.418318 19440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "fa4d8ccd744d456daab01f766fa8984a"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "c8bebabbfb9a229646f306c3cc1787fc"
server_key_iv: "2e70d5926c3abe85fd693fb3ed25838a"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.419464 19440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:15.420104 19471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.420253 19440 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:15.420336 19440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "fa4d8ccd744d456daab01f766fa8984a"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "c8bebabbfb9a229646f306c3cc1787fc"
server_key_iv: "2e70d5926c3abe85fd693fb3ed25838a"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.420392 19440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:15.430821 19440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:15.431056 19440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:15.431172 19440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:15.434720 19440 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:35921
I20260812 14:46:15.434744 19573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:35921 every 8 connection(s)
I20260812 14:46:15.435034 19440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:15.435899 19575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:15.437878 19575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a: Bootstrap starting.
I20260812 14:46:15.438411 19575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:15.438655 19575 log.cc:824] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:15.439172 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 19440
I20260812 14:46:15.439291 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:15.439332 19575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a: No bootstrap required, opened a new log
I20260812 14:46:15.439491 16271 external_mini_cluster.cc:1428] Setting key e2949091d1b008bc6cd92ce9e63dadd6
I20260812 14:46:15.441072 19575 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4d8ccd744d456daab01f766fa8984a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35921 } }
I20260812 14:46:15.441216 19575 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:15.441253 19575 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa4d8ccd744d456daab01f766fa8984a, State: Initialized, Role: FOLLOWER
I20260812 14:46:15.441373 19575 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4d8ccd744d456daab01f766fa8984a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35921 } }
I20260812 14:46:15.441454 19575 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:15.441481 19575 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:15.441526 19575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:15.442075 19575 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4d8ccd744d456daab01f766fa8984a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35921 } }
I20260812 14:46:15.442171 19575 leader_election.cc:302] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa4d8ccd744d456daab01f766fa8984a; no voters:
I20260812 14:46:15.442333 19575 leader_election.cc:288] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:15.442368 19582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:15.442482 19582 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [term 1 LEADER]: Becoming Leader. State: Replica: fa4d8ccd744d456daab01f766fa8984a, State: Running, Role: LEADER
I20260812 14:46:15.442574 19575 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:15.442571 19582 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4d8ccd744d456daab01f766fa8984a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35921 } }
I20260812 14:46:15.443255 19583 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa4d8ccd744d456daab01f766fa8984a. Latest consensus state: current_term: 1 leader_uuid: "fa4d8ccd744d456daab01f766fa8984a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4d8ccd744d456daab01f766fa8984a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35921 } } }
I20260812 14:46:15.443331 19583 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:15.443240 19584 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa4d8ccd744d456daab01f766fa8984a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa4d8ccd744d456daab01f766fa8984a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 35921 } } }
I20260812 14:46:15.443506 19584 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:15.444140 19597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:15.444579 19597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:15.445914 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.439972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.15.227.254:35921)
0812 14:46:15.440142 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.440148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.440171 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:46:15.441044 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.441049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.441098 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.441198 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.441427 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.442876 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.444003 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.444574 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.444681 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.445283 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.445286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.445287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:15.445433 (+ 146us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.445465 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:15.445474 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.445529 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.445606 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.445682 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260812 14:46:15.446399 19597 catalog_manager.cc:1347] Generated new cluster ID: 2fe8aca7487d45f49eda86a1eae91bb2
I20260812 14:46:15.446452 19597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:15.472357 19597 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:15.472815 19597 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:15.474956 19597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a: Generated new TSK 0
I20260812 14:46:15.475095 19597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:15.476372 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35921
--builtin_ntp_servers=127.15.227.212:38525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:15.556177 19614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:15.556366 19614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:15.556386 19614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:15.557480 19614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:15.557518 19614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:15.557540 19614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:46:15.558852 19614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:15.559054 19614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:15.559321 19614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:15.564831 19628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.564828 19627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.565026 19636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:15.565130 19614 server_base.cc:1034] running on GCE node
I20260812 14:46:15.565320 19614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:15.565537 19614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:15.566677 19614 hybrid_clock.cc:648] HybridClock initialized: now 1786545975566661 us; error 34 us; skew 500 ppm
I20260812 14:46:15.567838 19614 webserver.cc:460] Webserver started at http://127.15.227.193:41733/ using document root <none> and password file <none>
I20260812 14:46:15.568039 19614 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:15.568081 19614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:15.568197 19614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:15.569149 19614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "282e1449db5a4ad087b770cce6949bdf"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "f31d5e8d615ca03c8b83786d12d9bffe"
server_key_iv: "55c556976451ed65212bcefea96c910e"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.569471 19614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "282e1449db5a4ad087b770cce6949bdf"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "f31d5e8d615ca03c8b83786d12d9bffe"
server_key_iv: "55c556976451ed65212bcefea96c910e"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.570716 19614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:46:15.571431 19652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.571589 19614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:15.571659 19614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "282e1449db5a4ad087b770cce6949bdf"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "f31d5e8d615ca03c8b83786d12d9bffe"
server_key_iv: "55c556976451ed65212bcefea96c910e"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.571727 19614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:15.605338 19614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:15.605593 19614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:15.605715 19614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:15.605912 19614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:15.606228 19614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:15.606262 19614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.606281 19614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:15.606304 19614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.611838 19614 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:33831
I20260812 14:46:15.611887 19877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:33831 every 8 connection(s)
I20260812 14:46:15.612164 19614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:46:15.616803 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.612873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:39909 (local address 127.15.227.254:35921)
0812 14:46:15.612925 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.612927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.613128 (+ 201us) server_negotiation.cc:407] Connection header received
0812 14:46:15.613594 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.613595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.613607 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.613661 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.614279 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.614763 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.615655 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.615841 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.615888 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.616587 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.616588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.616589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:15.616594 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.616608 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:15.616611 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.616654 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.616722 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.616746 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:46:15.616971 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.612845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:35921 (local address 127.15.227.193:39909)
0812 14:46:15.613017 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.613055 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.613511 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.613712 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.613715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.613725 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:15.614203 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.614208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.614828 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.614828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.615588 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.615590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.615901 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.615902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.616003 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.616449 (+ 446us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:15.616449 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:15.616483 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:15.616504 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:15.616666 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:15.616669 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:15.616671 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.616719 (+ 48us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.616794 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260812 14:46:15.617422 19878 heartbeater.cc:347] Connected to a master server at 127.15.227.254:35921
I20260812 14:46:15.617504 19878 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:15.617657 19878 heartbeater.cc:511] Master 127.15.227.254:35921 requested a full tablet report, sending...
I20260812 14:46:15.618160 19503 ts_manager.cc:194] Registered new tserver with Master: 282e1449db5a4ad087b770cce6949bdf (127.15.227.193:33831)
I20260812 14:46:15.618749 19503 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.193:39909
I20260812 14:46:15.621681 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 19614
I20260812 14:46:15.621778 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:46:15.621924 16271 external_mini_cluster.cc:1428] Setting key d93774a74b768a16a1a9524738f395d4
I20260812 14:46:15.622799 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35921
--builtin_ntp_servers=127.15.227.212:38525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:15.705156 19882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:15.705340 19882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:15.705368 19882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:15.706475 19882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:15.706511 19882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:15.706543 19882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:46:15.707832 19882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:15.708058 19882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:15.708334 19882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:15.712888 19890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.712888 19893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.712915 19891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:15.713020 19882 server_base.cc:1034] running on GCE node
I20260812 14:46:15.713302 19882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:15.713527 19882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:15.714707 19882 hybrid_clock.cc:648] HybridClock initialized: now 1786545975714692 us; error 30 us; skew 500 ppm
I20260812 14:46:15.715843 19882 webserver.cc:460] Webserver started at http://127.15.227.194:34857/ using document root <none> and password file <none>
I20260812 14:46:15.716060 19882 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:15.716109 19882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:15.716230 19882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:15.717187 19882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "543517b2dca242fd953e7d6e8967a4b7"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "028278ff2ec2026031c2f648e92d75df"
server_key_iv: "1c6eebae3034a5b8a752a49f0cc99e0c"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.717494 19882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "543517b2dca242fd953e7d6e8967a4b7"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "028278ff2ec2026031c2f648e92d75df"
server_key_iv: "1c6eebae3034a5b8a752a49f0cc99e0c"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.718710 19882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:46:15.719322 19903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.719468 19882 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:15.719545 19882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "543517b2dca242fd953e7d6e8967a4b7"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "028278ff2ec2026031c2f648e92d75df"
server_key_iv: "1c6eebae3034a5b8a752a49f0cc99e0c"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.719601 19882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:15.731844 19882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:15.732064 19882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:15.732177 19882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:15.732354 19882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:15.732620 19882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:15.732661 19882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.732686 19882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:15.732703 19882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.737874 19882 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:43545
I20260812 14:46:15.737933 20142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:43545 every 8 connection(s)
I20260812 14:46:15.738209 19882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:46:15.742794 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.738813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:47649 (local address 127.15.227.254:35921)
0812 14:46:15.738855 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.738857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.739051 (+ 194us) server_negotiation.cc:407] Connection header received
0812 14:46:15.739488 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.739490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.739503 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.739554 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.740158 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.740651 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.741566 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.741771 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.741827 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.742529 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.742531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.742531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:15.742536 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.742551 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:15.742556 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.742600 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.742676 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.742706 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:46:15.742976 20145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.738788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:35921 (local address 127.15.227.194:47649)
0812 14:46:15.738940 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.738981 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.739392 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.739616 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.739620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.739629 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:15.740071 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.740075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.740733 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.740735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.741483 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.741485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.741863 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.741864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.741994 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.742405 (+ 411us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:15.742405 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:15.742429 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:15.742450 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:15.742606 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:15.742612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:15.742615 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.742675 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.742787 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
I20260812 14:46:15.743494 20143 heartbeater.cc:347] Connected to a master server at 127.15.227.254:35921
I20260812 14:46:15.743582 20143 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:15.743726 20143 heartbeater.cc:511] Master 127.15.227.254:35921 requested a full tablet report, sending...
I20260812 14:46:15.744014 19503 ts_manager.cc:194] Registered new tserver with Master: 543517b2dca242fd953e7d6e8967a4b7 (127.15.227.194:43545)
I20260812 14:46:15.744462 19503 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.194:47649
I20260812 14:46:15.748056 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 19882
I20260812 14:46:15.748152 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:46:15.748308 16271 external_mini_cluster.cc:1428] Setting key 28a852d504e8284a1be8dc62c3075ff5
I20260812 14:46:15.749089 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35921
--builtin_ntp_servers=127.15.227.212:38525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:15.827734 20146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:15.827912 20146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:15.827941 20146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:15.829017 20146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:15.829053 20146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:15.829087 20146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:46:15.830400 20146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:38525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:35921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:15.830629 20146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:15.830878 20146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:46:15.835460 20146 server_base.cc:1034] running on GCE node
W20260812 14:46:15.835546 20161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.835584 20159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:15.835552 20165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:15.835807 20146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:15.835997 20146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:15.837126 20146 hybrid_clock.cc:648] HybridClock initialized: now 1786545975837114 us; error 27 us; skew 500 ppm
I20260812 14:46:15.838246 20146 webserver.cc:460] Webserver started at http://127.15.227.195:35367/ using document root <none> and password file <none>
I20260812 14:46:15.838443 20146 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:15.838486 20146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:15.838603 20146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:15.839560 20146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "9233364232af4cf7a271ce5fb503f45f"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "455e0e65b1c6c6e1560097638ac1ba38"
server_key_iv: "fb15dbeb8580f29809241db489dbf63c"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.839865 20146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "9233364232af4cf7a271ce5fb503f45f"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "455e0e65b1c6c6e1560097638ac1ba38"
server_key_iv: "fb15dbeb8580f29809241db489dbf63c"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.841005 20146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:46:15.841605 20178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.841737 20146 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:15.841816 20146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "9233364232af4cf7a271ce5fb503f45f"
format_stamp: "Formatted at 2026-08-12 14:46:15 on dist-test-slave-q9h9"
server_key: "455e0e65b1c6c6e1560097638ac1ba38"
server_key_iv: "fb15dbeb8580f29809241db489dbf63c"
server_key_version: "encryptionkey@0"
I20260812 14:46:15.841876 20146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:15.854975 20146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:15.855197 20146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:15.855310 20146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:15.855494 20146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:15.855767 20146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:15.855800 20146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.855818 20146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:15.855841 20146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:15.860981 20146 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:45293
I20260812 14:46:15.861044 20392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:45293 every 8 connection(s)
I20260812 14:46:15.861299 20146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:46:15.863727 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 20146
I20260812 14:46:15.863817 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:46:15.863991 16271 external_mini_cluster.cc:1428] Setting key 6f74244f9bececcb7c2abd49a0eb9012
I20260812 14:46:15.866217 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.861919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:52921 (local address 127.15.227.254:35921)
0812 14:46:15.861977 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.861979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.862203 (+ 224us) server_negotiation.cc:407] Connection header received
0812 14:46:15.862587 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.862588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.862599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.862656 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.863337 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.863889 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.865034 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.865236 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.865289 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.865938 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.865939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.865940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:15.865944 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.865959 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:15.865963 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.866008 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.866106 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.866137 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260812 14:46:15.866405 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.861905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:35921 (local address 127.15.227.195:52921)
0812 14:46:15.862099 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.862130 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.862511 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.862708 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.862711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.862719 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:15.863270 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.863275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.863981 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.863983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.864961 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.864964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.865302 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.865303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.865434 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.865820 (+ 386us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:15.865820 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:15.865844 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:15.865865 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:15.866032 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:15.866037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:15.866040 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.866103 (+ 63us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.866197 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260812 14:46:15.866822 20393 heartbeater.cc:347] Connected to a master server at 127.15.227.254:35921
I20260812 14:46:15.866910 20393 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:15.867080 20393 heartbeater.cc:511] Master 127.15.227.254:35921 requested a full tablet report, sending...
I20260812 14:46:15.867408 19503 ts_manager.cc:194] Registered new tserver with Master: 9233364232af4cf7a271ce5fb503f45f (127.15.227.195:45293)
I20260812 14:46:15.867741 19503 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.195:52921
I20260812 14:46:15.875490 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:46:15.879516 19581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.876776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.15.227.254:35921)
0812 14:46:15.876841 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.876843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.876949 (+ 106us) server_negotiation.cc:407] Connection header received
0812 14:46:15.877005 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.877006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.877017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.877056 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:15.877220 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.877668 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.878459 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.878641 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.878696 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.879252 (+ 556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:15.879254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:15.879254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:15.879258 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:15.879271 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:15.879274 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:15.879347 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.879433 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.879443 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 14:46:15.880867 19503 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:46:15.882405 19503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:46:15.888197 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.884341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33831 (local address 127.0.0.1:57000)
0812 14:46:15.884584 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.884592 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.884629 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.885163 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.885164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.885166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.885199 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.885200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.886372 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.886373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.887111 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.887114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.888001 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.888001 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.888057 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.888067 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.888113 (+ 46us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.888148 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":63,"threads_started":1}
I20260812 14:46:15.888258 20418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.884579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57000 (local address 127.15.227.193:33831)
0812 14:46:15.885009 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.885013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.885022 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:46:15.885056 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.885058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.885072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.885158 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.885257 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.886313 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.887213 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.887926 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.888005 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.888091 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.888125 (+ 34us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.888163 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":94,"threads_started":1}
I20260812 14:46:15.888443 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.884334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:45293 (local address 127.0.0.1:35802)
0812 14:46:15.884516 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.884552 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.884613 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.884961 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.884963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.884965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.885018 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.885020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.886292 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.886292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.887404 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.887405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.888244 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.888245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.888305 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.888308 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.888360 (+ 52us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.888396 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":5,"thread_start_us":80,"threads_started":1}
I20260812 14:46:15.888495 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.884388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35802 (local address 127.15.227.195:45293)
0812 14:46:15.884823 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.884827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.884833 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:46:15.884858 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.884859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.884870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.884944 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.885103 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.886215 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.887486 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.888156 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.888228 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.888313 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.888389 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.888425 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":85,"threads_started":1}
I20260812 14:46:15.889134 20272 tablet_service.cc:1467] Processing CreateTablet for tablet 223d00a5577843c6b73444b62b230a00 (DEFAULT_TABLE table=test-table [id=ed33be28859e4ad1978d470bec4d8567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:15.889297 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.884352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:43545 (local address 127.0.0.1:59544)
0812 14:46:15.884707 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.884714 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.884747 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.884928 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.884931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.884933 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.884997 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.884999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.886160 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.886161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.887398 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.887401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.889068 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.889069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.889129 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.889133 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.889188 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.889228 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":8,"thread_start_us":91,"threads_started":1}
I20260812 14:46:15.889320 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.884582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59544 (local address 127.15.227.194:43545)
0812 14:46:15.884761 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.884765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.884776 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:46:15.884810 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.884812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.884824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.884892 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.885072 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.886086 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.888353 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.888987 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.889054 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.889143 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.889203 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.889244 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":79,"threads_started":1}
I20260812 14:46:15.889465 20272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 223d00a5577843c6b73444b62b230a00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:15.891949 20423 tablet_bootstrap.cc:492] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Bootstrap starting.
I20260812 14:46:15.892244 20013 tablet_service.cc:1467] Processing CreateTablet for tablet 223d00a5577843c6b73444b62b230a00 (DEFAULT_TABLE table=test-table [id=ed33be28859e4ad1978d470bec4d8567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:15.892529 20013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 223d00a5577843c6b73444b62b230a00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:15.892581 20423 tablet_bootstrap.cc:654] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:15.892902 20423 log.cc:824] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:15.893566 20423 tablet_bootstrap.cc:492] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: No bootstrap required, opened a new log
I20260812 14:46:15.893621 20423 ts_tablet_manager.cc:1397] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:15.895114 20433 tablet_bootstrap.cc:492] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Bootstrap starting.
I20260812 14:46:15.895578 19739 tablet_service.cc:1467] Processing CreateTablet for tablet 223d00a5577843c6b73444b62b230a00 (DEFAULT_TABLE table=test-table [id=ed33be28859e4ad1978d470bec4d8567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:15.895692 20433 tablet_bootstrap.cc:654] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:15.895862 19739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 223d00a5577843c6b73444b62b230a00. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:15.895955 20433 log.cc:824] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:15.896675 20433 tablet_bootstrap.cc:492] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: No bootstrap required, opened a new log
I20260812 14:46:15.896741 20433 ts_tablet_manager.cc:1397] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:15.898324 20436 tablet_bootstrap.cc:492] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf: Bootstrap starting.
I20260812 14:46:15.898528 20433 raft_consensus.cc:348] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.898654 20433 raft_consensus.cc:374] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:15.898686 20433 raft_consensus.cc:729] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543517b2dca242fd953e7d6e8967a4b7, State: Initialized, Role: FOLLOWER
I20260812 14:46:15.898622 20423 raft_consensus.cc:348] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.898793 20423 raft_consensus.cc:374] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:15.898837 20423 raft_consensus.cc:729] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9233364232af4cf7a271ce5fb503f45f, State: Initialized, Role: FOLLOWER
I20260812 14:46:15.898831 20433 consensus_queue.cc:260] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.898954 20436 tablet_bootstrap.cc:654] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:15.898980 20423 consensus_queue.cc:260] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.899078 20433 ts_tablet_manager.cc:1428] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:46:15.899188 20143 heartbeater.cc:503] Master 127.15.227.254:35921 was elected leader, sending a full tablet report...
I20260812 14:46:15.899255 20423 ts_tablet_manager.cc:1428] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260812 14:46:15.899267 20436 log.cc:824] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:15.899387 20393 heartbeater.cc:503] Master 127.15.227.254:35921 was elected leader, sending a full tablet report...
I20260812 14:46:15.900133 20436 tablet_bootstrap.cc:492] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf: No bootstrap required, opened a new log
I20260812 14:46:15.900203 20436 ts_tablet_manager.cc:1397] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:15.902262 20436 raft_consensus.cc:348] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.902379 20436 raft_consensus.cc:374] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:15.902397 20436 raft_consensus.cc:729] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 282e1449db5a4ad087b770cce6949bdf, State: Initialized, Role: FOLLOWER
I20260812 14:46:15.902500 20436 consensus_queue.cc:260] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.902737 20436 ts_tablet_manager.cc:1428] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:46:15.902801 19878 heartbeater.cc:503] Master 127.15.227.254:35921 was elected leader, sending a full tablet report...
I20260812 14:46:15.913050 20440 raft_consensus.cc:482] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:15.913118 20440 raft_consensus.cc:504] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.913372 20440 leader_election.cc:288] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9233364232af4cf7a271ce5fb503f45f (127.15.227.195:45293), 282e1449db5a4ad087b770cce6949bdf (127.15.227.193:33831)
I20260812 14:46:15.916800 20418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.913553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:33745 (local address 127.15.227.193:33831)
0812 14:46:15.913593 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.913595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.913722 (+ 127us) server_negotiation.cc:407] Connection header received
0812 14:46:15.913745 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.913746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.913755 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.913799 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.913915 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.914602 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.915961 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.916482 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.916555 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.916563 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.916704 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.916743 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:46:15.916800 20416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.913532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:37359 (local address 127.15.227.195:45293)
0812 14:46:15.913580 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.913581 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.913591 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:46:15.913670 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.913671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.913679 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.913733 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:15.913870 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.914603 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.915961 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.916503 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.916559 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.916564 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.916704 (+ 140us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.916738 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:46:15.916826 20145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.913492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:45293 (local address 127.15.227.194:37359)
0812 14:46:15.913558 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.913568 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.913616 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.913742 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.913743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.913744 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.913803 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.913805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.914668 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.914668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.915886 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.915889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.916576 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.916577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.916643 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.916645 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.916694 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.916738 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":73}
I20260812 14:46:15.916832 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.913509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33831 (local address 127.15.227.194:33745)
0812 14:46:15.913650 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.913658 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:15.913706 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:15.913810 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:15.913811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:15.913812 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:15.913861 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.913862 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.914668 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.914668 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.915891 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:15.915893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.916556 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.916557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:15.916634 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.916639 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:46:15.916694 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:46:15.916736 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":28,"thread_start_us":98,"threads_started":1}
I20260812 14:46:15.917109 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d00a5577843c6b73444b62b230a00" candidate_uuid: "543517b2dca242fd953e7d6e8967a4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9233364232af4cf7a271ce5fb503f45f" is_pre_election: true
I20260812 14:46:15.917199 19772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d00a5577843c6b73444b62b230a00" candidate_uuid: "543517b2dca242fd953e7d6e8967a4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282e1449db5a4ad087b770cce6949bdf" is_pre_election: true
I20260812 14:46:15.917243 20314 raft_consensus.cc:2454] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543517b2dca242fd953e7d6e8967a4b7 in term 0.
I20260812 14:46:15.917306 19772 raft_consensus.cc:2454] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 543517b2dca242fd953e7d6e8967a4b7 in term 0.
I20260812 14:46:15.917451 19908 leader_election.cc:302] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 543517b2dca242fd953e7d6e8967a4b7, 9233364232af4cf7a271ce5fb503f45f; no voters:
I20260812 14:46:15.917608 20440 raft_consensus.cc:2781] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:46:15.917647 20440 raft_consensus.cc:482] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:46:15.917670 20440 raft_consensus.cc:3037] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:15.918293 20440 raft_consensus.cc:504] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.918398 20440 leader_election.cc:288] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [CANDIDATE]: Term 1 election: Requested vote from peers 9233364232af4cf7a271ce5fb503f45f (127.15.227.195:45293), 282e1449db5a4ad087b770cce6949bdf (127.15.227.193:33831)
I20260812 14:46:15.918538 20314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d00a5577843c6b73444b62b230a00" candidate_uuid: "543517b2dca242fd953e7d6e8967a4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9233364232af4cf7a271ce5fb503f45f"
I20260812 14:46:15.918612 20314 raft_consensus.cc:3037] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:15.919229 20314 raft_consensus.cc:2454] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543517b2dca242fd953e7d6e8967a4b7 in term 1.
I20260812 14:46:15.919323 19772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "223d00a5577843c6b73444b62b230a00" candidate_uuid: "543517b2dca242fd953e7d6e8967a4b7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "282e1449db5a4ad087b770cce6949bdf"
I20260812 14:46:15.919366 19908 leader_election.cc:302] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 543517b2dca242fd953e7d6e8967a4b7, 9233364232af4cf7a271ce5fb503f45f; no voters:
I20260812 14:46:15.919384 19772 raft_consensus.cc:3037] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:15.919458 20440 raft_consensus.cc:2781] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:15.919626 20440 raft_consensus.cc:686] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 1 LEADER]: Becoming Leader. State: Replica: 543517b2dca242fd953e7d6e8967a4b7, State: Running, Role: LEADER
I20260812 14:46:15.919710 20440 consensus_queue.cc:237] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } }
I20260812 14:46:15.920159 19772 raft_consensus.cc:2454] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 543517b2dca242fd953e7d6e8967a4b7 in term 1.
I20260812 14:46:15.920462 19503 catalog_manager.cc:5495] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 543517b2dca242fd953e7d6e8967a4b7 (127.15.227.194). New cstate: current_term: 1 leader_uuid: "543517b2dca242fd953e7d6e8967a4b7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "543517b2dca242fd953e7d6e8967a4b7" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 43545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 } health_report { overall_health: UNKNOWN } } }
I20260812 14:46:15.933066 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.930944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.15.227.194:43545)
0812 14:46:15.930987 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:15.930989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:15.930996 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:15.931042 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:15.931043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:15.931060 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:15.931140 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:15.931251 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.931673 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.932370 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:15.932545 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:15.932597 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:15.932767 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:15.932846 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:15.932925 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:15.932985 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 14:46:15.932992 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:46:15.934523 20314 raft_consensus.cc:1261] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 1 FOLLOWER]: Refusing update from remote peer 543517b2dca242fd953e7d6e8967a4b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:15.934543 19772 raft_consensus.cc:1261] T 223d00a5577843c6b73444b62b230a00 P 282e1449db5a4ad087b770cce6949bdf [term 1 FOLLOWER]: Refusing update from remote peer 543517b2dca242fd953e7d6e8967a4b7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:15.934831 20440 consensus_queue.cc:1035] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9233364232af4cf7a271ce5fb503f45f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 45293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:15.934950 20451 consensus_queue.cc:1035] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "282e1449db5a4ad087b770cce6949bdf" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 33831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:15.936952 20455 mvcc.cc:205] Tried to move back new op lower bound from 7317692317425758208 to 7317692317367676928. Current Snapshot: MvccSnapshot[applied={T|T < 7317692317425758208}]
I20260812 14:46:15.937064 20458 mvcc.cc:205] Tried to move back new op lower bound from 7317692317425758208 to 7317692317367676928. Current Snapshot: MvccSnapshot[applied={T|T < 7317692317425758208}]
I20260812 14:46:15.937780 20457 mvcc.cc:205] Tried to move back new op lower bound from 7317692317425758208 to 7317692317367676928. Current Snapshot: MvccSnapshot[applied={T|T < 7317692317425758208 or (T in {7317692317425758208})}]
I20260812 14:46:15.939332 19503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:15.939409 19503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:15.940002 19503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa4d8ccd744d456daab01f766fa8984a: Sending DeleteTablet for 3 replicas of tablet 223d00a5577843c6b73444b62b230a00
I20260812 14:46:15.940294 20272 tablet_service.cc:1514] Processing DeleteTablet for tablet 223d00a5577843c6b73444b62b230a00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:15 UTC) from {username='slave'} at 127.0.0.1:35802
I20260812 14:46:15.940327 20013 tablet_service.cc:1514] Processing DeleteTablet for tablet 223d00a5577843c6b73444b62b230a00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:15 UTC) from {username='slave'} at 127.0.0.1:59544
I20260812 14:46:15.940526 20482 tablet_replica.cc:330] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: stopping tablet replica
I20260812 14:46:15.940620 20482 raft_consensus.cc:2229] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:15.940621 19739 tablet_service.cc:1514] Processing DeleteTablet for tablet 223d00a5577843c6b73444b62b230a00 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:15 UTC) from {username='slave'} at 127.0.0.1:57000
I20260812 14:46:15.940725 20482 raft_consensus.cc:2258] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:15.940855 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 19614
I20260812 14:46:15.940886 20480 tablet_replica.cc:330] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: stopping tablet replica
I20260812 14:46:15.940969 20480 raft_consensus.cc:2229] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:46:15.941054 20480 raft_consensus.cc:2258] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:15.951660 20482 ts_tablet_manager.cc:1905] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:15.953855 20482 ts_tablet_manager.cc:1918] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:15.953913 20482 log.cc:1196] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/223d00a5577843c6b73444b62b230a00
I20260812 14:46:15.954118 20482 ts_tablet_manager.cc:1939] T 223d00a5577843c6b73444b62b230a00 P 9233364232af4cf7a271ce5fb503f45f: Deleting consensus metadata
I20260812 14:46:15.956092 20480 ts_tablet_manager.cc:1905] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:15.957124 20480 ts_tablet_manager.cc:1918] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:15.957173 20480 log.cc:1196] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/223d00a5577843c6b73444b62b230a00
I20260812 14:46:15.957346 20480 ts_tablet_manager.cc:1939] T 223d00a5577843c6b73444b62b230a00 P 543517b2dca242fd953e7d6e8967a4b7: Deleting consensus metadata
I20260812 14:46:15.957681 19478 catalog_manager.cc:4841] TS 543517b2dca242fd953e7d6e8967a4b7 (127.15.227.194:43545): tablet 223d00a5577843c6b73444b62b230a00 (table test-table [id=ed33be28859e4ad1978d470bec4d8567]) successfully deleted
I20260812 14:46:15.959038 19475 catalog_manager.cc:4841] TS 9233364232af4cf7a271ce5fb503f45f (127.15.227.195:45293): tablet 223d00a5577843c6b73444b62b230a00 (table test-table [id=ed33be28859e4ad1978d470bec4d8567]) successfully deleted
W20260812 14:46:15.965931 19477 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:33831 (error 108)
I20260812 14:46:15.966195 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 19882
I20260812 14:46:15.969925 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:15.966329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:33831 (local address 127.0.0.1:57008)
0812 14:46:15.969844 (+ 3515us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:15.969893 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3490}
W20260812 14:46:15.970098 19477 catalog_manager.cc:4584] TS 282e1449db5a4ad087b770cce6949bdf (127.15.227.193:33831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 223d00a5577843c6b73444b62b230a00: Network error: Client connection negotiation failed: client connection to 127.15.227.193:33831: connect: Connection refused (error 111)
I20260812 14:46:15.980592 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 20146
I20260812 14:46:15.996019 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 19440
2026-08-12T14:46:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (704 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:16 dist-test-slave-q9h9 krb5kdc[20490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:16 dist-test-slave-q9h9 krb5kdc[20490](info): set up 2 sockets
Aug 12 14:46:16 dist-test-slave-q9h9 krb5kdc[20490](info): commencing operation
krb5kdc: starting...
W20260812 14:46:17.180537 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:17.291870 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36657
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:36657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:17.370035 20508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:17.370227 20508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:17.370258 20508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:17.371356 20508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:17.371395 20508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:17.371414 20508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:17.371430 20508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:17.371446 20508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:17.372910 20508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:36657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:36657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:17.373142 20508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:17.373338 20508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:17.377938 20521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:17.377913 20519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:17.378175 20508 server_base.cc:1034] running on GCE node
W20260812 14:46:17.377923 20518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:17.378454 20508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:17.378721 20508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:17.379856 20508 hybrid_clock.cc:648] HybridClock initialized: now 1786545977379845 us; error 31 us; skew 500 ppm
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:17.382186 20508 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:17.382614 20508 webserver.cc:460] Webserver started at http://127.15.227.254:43823/ using document root <none> and password file <none>
I20260812 14:46:17.382824 20508 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:17.382871 20508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:17.382997 20508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:17.383951 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "4bcf76afe12e4be98dc7328132586d44"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "5bf23c8c6e17131d7d7e84f8db94b8ac"
server_key_iv: "651ab8f391b71b9fd86869540fba4b22"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.384284 20508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "4bcf76afe12e4be98dc7328132586d44"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "5bf23c8c6e17131d7d7e84f8db94b8ac"
server_key_iv: "651ab8f391b71b9fd86869540fba4b22"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.385497 20508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:46:17.386198 20533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.386349 20508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:17.386435 20508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "4bcf76afe12e4be98dc7328132586d44"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "5bf23c8c6e17131d7d7e84f8db94b8ac"
server_key_iv: "651ab8f391b71b9fd86869540fba4b22"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.386502 20508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:17.394742 20508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:17.396400 20508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:17.396520 20508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:17.400472 20508 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:36657
I20260812 14:46:17.400506 20645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:36657 every 8 connection(s)
I20260812 14:46:17.400806 20508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:17.401566 20646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:17.403767 20646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44: Bootstrap starting.
I20260812 14:46:17.404302 20646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:17.404552 20646 log.cc:824] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:17.405186 20646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44: No bootstrap required, opened a new log
I20260812 14:46:17.405834 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 20508
I20260812 14:46:17.405936 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:17.406141 16271 external_mini_cluster.cc:1428] Setting key 71d816a6443d39375754aed2f1be9286
I20260812 14:46:17.407167 20646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcf76afe12e4be98dc7328132586d44" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36657 } }
I20260812 14:46:17.407325 20646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:17.407359 20646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bcf76afe12e4be98dc7328132586d44, State: Initialized, Role: FOLLOWER
I20260812 14:46:17.407478 20646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcf76afe12e4be98dc7328132586d44" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36657 } }
I20260812 14:46:17.407560 20646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:17.407584 20646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:17.407631 20646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:17.408593 20646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcf76afe12e4be98dc7328132586d44" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36657 } }
I20260812 14:46:17.408699 20646 leader_election.cc:302] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bcf76afe12e4be98dc7328132586d44; no voters:
I20260812 14:46:17.408839 20646 leader_election.cc:288] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:17.408874 20655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:17.409024 20655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [term 1 LEADER]: Becoming Leader. State: Replica: 4bcf76afe12e4be98dc7328132586d44, State: Running, Role: LEADER
I20260812 14:46:17.409055 20646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:17.409140 20655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcf76afe12e4be98dc7328132586d44" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36657 } }
I20260812 14:46:17.410112 20657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bcf76afe12e4be98dc7328132586d44. Latest consensus state: current_term: 1 leader_uuid: "4bcf76afe12e4be98dc7328132586d44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcf76afe12e4be98dc7328132586d44" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36657 } } }
I20260812 14:46:17.410193 20657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:17.410497 20659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bcf76afe12e4be98dc7328132586d44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bcf76afe12e4be98dc7328132586d44" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 36657 } } }
I20260812 14:46:17.410566 20659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44 [sys.catalog]: This master's current role is: LEADER
W20260812 14:46:17.410681 20687 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:46:17.410717 20687 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:46:17.410776 20692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:17.410909 20692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:17.413472 20692 catalog_manager.cc:1347] Generated new cluster ID: cda0eb930b9540cdb44a2b13a76e7b8f
I20260812 14:46:17.413528 20692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:17.416316 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.406578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.15.227.254:36657)
0812 14:46:17.406768 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.406777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.406810 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:46:17.407141 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.407149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.407208 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.407350 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:17.407993 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.409313 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.410335 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.410933 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.411055 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.413268 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:17.413272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:17.413273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:17.413283 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:17.414812 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.415129 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.415132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.415133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.415233 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.415405 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.415406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.415407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.415631 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:17.415767 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.415872 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.415999 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260812 14:46:17.446488 20692 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:17.447185 20692 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:17.455626 20692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44: Generated new TSK 0
I20260812 14:46:17.455786 20692 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.227.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
Entry for principal kudu/127.15.227.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab.
I20260812 14:46:17.481357 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36657
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:17.560673 20701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:17.560858 20701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:17.560887 20701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:17.561975 20701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:17.562033 20701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:17.562071 20701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:46:17.563442 20701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.193.keytab
--principal=kudu/127.15.227.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:17.563669 20701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:17.563864 20701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:17.568471 20708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:17.568537 20715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:17.568602 20701 server_base.cc:1034] running on GCE node
W20260812 14:46:17.568475 20710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:17.568854 20701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:17.569077 20701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:17.570204 20701 hybrid_clock.cc:648] HybridClock initialized: now 1786545977570187 us; error 35 us; skew 500 ppm
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:17.572465 20701 init.cc:377] Logged in from keytab as kudu/127.15.227.193@KRBTEST.COM (short username kudu)
I20260812 14:46:17.572842 20701 webserver.cc:460] Webserver started at http://127.15.227.193:40157/ using document root <none> and password file <none>
I20260812 14:46:17.573025 20701 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:17.573072 20701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:17.573194 20701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:17.574162 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "d6473ba4c058421dbb8e7c20bcddb569"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "b31c424919f8513ed8fea9b14b8d9c7c"
server_key_iv: "7bdcc1c83311359c6d0fe9c15775bce7"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.574477 20701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "d6473ba4c058421dbb8e7c20bcddb569"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "b31c424919f8513ed8fea9b14b8d9c7c"
server_key_iv: "7bdcc1c83311359c6d0fe9c15775bce7"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.575678 20701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:46:17.576315 20727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.576462 20701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:17.576541 20701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "d6473ba4c058421dbb8e7c20bcddb569"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "b31c424919f8513ed8fea9b14b8d9c7c"
server_key_iv: "7bdcc1c83311359c6d0fe9c15775bce7"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.576597 20701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:17.592985 20701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:17.594564 20701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:17.594686 20701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:17.594898 20701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:17.595162 20701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:17.595196 20701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.595232 20701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:17.595255 20701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.600442 20701 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:37915
I20260812 14:46:17.600484 20936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:37915 every 8 connection(s)
I20260812 14:46:17.600745 20701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:46:17.605948 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 20701
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.193@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:17.606077 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:46:17.606266 16271 external_mini_cluster.cc:1428] Setting key 9936686333d27b14f2d4839b61a7b656
I20260812 14:46:17.609043 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.601347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:52649 (local address 127.15.227.254:36657)
0812 14:46:17.601406 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.601408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.601572 (+ 164us) server_negotiation.cc:407] Connection header received
0812 14:46:17.601970 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.601971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.601979 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.602057 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:17.602884 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.603418 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.604325 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.604515 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.604555 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.606479 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:17.606480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:17.606481 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:17.606488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:17.607803 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.608159 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.608160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.608161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.608218 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.608431 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.608432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.608432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.608535 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:17.608592 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.608868 (+ 276us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.608954 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:46:17.609254 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.601317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36657 (local address 127.15.227.193:52649)
0812 14:46:17.601465 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:17.601502 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:17.601903 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:17.602106 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:17.602109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:17.602619 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:17.602798 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.602803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.603498 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.603499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.604263 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.604266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.604564 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.604565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.604662 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.605070 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:17.605078 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:17.606394 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:17.607881 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:17.607889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:17.607891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:17.608092 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:17.608274 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:17.608275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:17.608276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:17.608375 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:17.608593 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:17.608601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:17.608727 (+ 126us) client_negotiation.cc:769] Sending connection context
0812 14:46:17.608811 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 14:46:17.608956 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260812 14:46:17.609772 20940 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36657
I20260812 14:46:17.609856 20940 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:17.610050 20940 heartbeater.cc:511] Master 127.15.227.254:36657 requested a full tablet report, sending...
I20260812 14:46:17.610548 20576 ts_manager.cc:194] Registered new tserver with Master: d6473ba4c058421dbb8e7c20bcddb569 (127.15.227.193:37915)
I20260812 14:46:17.611133 20576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.193@KRBTEST.COM'} at 127.15.227.193:52649
WARNING: no policy specified for kudu/127.15.227.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
Entry for principal kudu/127.15.227.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab.
I20260812 14:46:17.628473 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36657
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:17.707122 20949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:17.707298 20949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:17.707320 20949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:17.708400 20949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:17.708441 20949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:17.708467 20949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:46:17.709951 20949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.194.keytab
--principal=kudu/127.15.227.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:17.710175 20949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:17.710379 20949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:17.715026 20967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:17.715093 20971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:17.715060 20965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:17.715246 20949 server_base.cc:1034] running on GCE node
I20260812 14:46:17.715440 20949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:17.715659 20949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:17.716789 20949 hybrid_clock.cc:648] HybridClock initialized: now 1786545977716777 us; error 27 us; skew 500 ppm
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:17.719223 20949 init.cc:377] Logged in from keytab as kudu/127.15.227.194@KRBTEST.COM (short username kudu)
I20260812 14:46:17.719573 20949 webserver.cc:460] Webserver started at http://127.15.227.194:35103/ using document root <none> and password file <none>
I20260812 14:46:17.719741 20949 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:17.719780 20949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:17.719869 20949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:17.720780 20949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "c598f39887254b5d8da360b1bf3cd0a0"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "93e1e4cdf0c4cdf6423f92236f70713c"
server_key_iv: "50346f9b5812e8a1e78bae5e4a7e78df"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.721060 20949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "c598f39887254b5d8da360b1bf3cd0a0"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "93e1e4cdf0c4cdf6423f92236f70713c"
server_key_iv: "50346f9b5812e8a1e78bae5e4a7e78df"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.722293 20949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:17.722918 20982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.723070 20949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.723152 20949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "c598f39887254b5d8da360b1bf3cd0a0"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "93e1e4cdf0c4cdf6423f92236f70713c"
server_key_iv: "50346f9b5812e8a1e78bae5e4a7e78df"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.723204 20949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:17.732419 20949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:17.734318 20949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:17.734416 20949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:17.734563 20949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:17.734819 20949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:17.734849 20949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.734874 20949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:17.734890 20949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.740027 20949 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:35633
I20260812 14:46:17.740062 21197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:35633 every 8 connection(s)
I20260812 14:46:17.740311 20949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:46:17.742947 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 20949
I20260812 14:46:17.743062 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:46:17.743243 16271 external_mini_cluster.cc:1428] Setting key b9cbcee7daeee7dc6815b809455a5b16
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.194@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:17.748581 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.740885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:36765 (local address 127.15.227.254:36657)
0812 14:46:17.740947 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.740950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.741103 (+ 153us) server_negotiation.cc:407] Connection header received
0812 14:46:17.741533 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.741534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.741547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.741597 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:17.742475 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.743088 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.744106 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.744341 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.744410 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.746200 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:17.746202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:17.746202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:17.746210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:17.747387 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.747740 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.747742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.747742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.747797 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.748004 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.748005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.748005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.748109 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:17.748173 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.748407 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.748494 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:46:17.748718 21205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.740838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36657 (local address 127.15.227.194:36765)
0812 14:46:17.740993 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:17.741029 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:17.741444 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:17.741659 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:17.741662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:17.742207 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:17.742382 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.742386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.743168 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.743169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.744023 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.744025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.744409 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.744410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.744496 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.744882 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:17.744889 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:17.746118 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:17.747460 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:17.747467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:17.747469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:17.747676 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:17.747850 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:17.747851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:17.747851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:17.747949 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:17.748165 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:17.748171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:17.748295 (+ 124us) client_negotiation.cc:769] Sending connection context
0812 14:46:17.748378 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 14:46:17.748515 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260812 14:46:17.749156 21198 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36657
I20260812 14:46:17.749253 21198 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:17.749428 21198 heartbeater.cc:511] Master 127.15.227.254:36657 requested a full tablet report, sending...
I20260812 14:46:17.749792 20576 ts_manager.cc:194] Registered new tserver with Master: c598f39887254b5d8da360b1bf3cd0a0 (127.15.227.194:35633)
I20260812 14:46:17.750300 20576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.194@KRBTEST.COM'} at 127.15.227.194:36765
WARNING: no policy specified for kudu/127.15.227.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
Entry for principal kudu/127.15.227.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab.
I20260812 14:46:17.765552 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36657
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:17.843541 21208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:17.843713 21208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:17.843735 21208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:17.844828 21208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:17.844868 21208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:17.844897 21208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:46:17.846329 21208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.195.keytab
--principal=kudu/127.15.227.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:36657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:17.846527 21208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:17.846719 21208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:17.851285 21215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:17.851375 21217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:17.851291 21221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:17.851508 21208 server_base.cc:1034] running on GCE node
I20260812 14:46:17.851679 21208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:17.851900 21208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:17.853047 21208 hybrid_clock.cc:648] HybridClock initialized: now 1786545977853031 us; error 32 us; skew 500 ppm
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:17.855366 21208 init.cc:377] Logged in from keytab as kudu/127.15.227.195@KRBTEST.COM (short username kudu)
I20260812 14:46:17.855866 21208 webserver.cc:460] Webserver started at http://127.15.227.195:37139/ using document root <none> and password file <none>
I20260812 14:46:17.856061 21208 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:17.856130 21208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:17.856263 21208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:17.857234 21208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "de0da8dd429846e3a26bab8dbf3789a9"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "dc982c1757ce9359d7f2474ead4cc871"
server_key_iv: "ec0140acf537c91de8d230b23e71a2d5"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.857553 21208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "de0da8dd429846e3a26bab8dbf3789a9"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "dc982c1757ce9359d7f2474ead4cc871"
server_key_iv: "ec0140acf537c91de8d230b23e71a2d5"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.858791 21208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:46:17.859447 21233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.859588 21208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:17.859665 21208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "de0da8dd429846e3a26bab8dbf3789a9"
format_stamp: "Formatted at 2026-08-12 14:46:17 on dist-test-slave-q9h9"
server_key: "dc982c1757ce9359d7f2474ead4cc871"
server_key_iv: "ec0140acf537c91de8d230b23e71a2d5"
server_key_version: "encryptionkey@0"
I20260812 14:46:17.859730 21208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:17.874828 21208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:17.876600 21208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:17.876710 21208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:17.876906 21208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:17.877174 21208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:17.877214 21208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.877235 21208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:17.877259 21208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:17.883133 21208 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:42441
I20260812 14:46:17.883172 21445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:42441 every 8 connection(s)
I20260812 14:46:17.883440 21208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
Aug 12 14:46:17 dist-test-slave-q9h9 krb5kdc[20490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545977, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.195@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:17.889493 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21208
I20260812 14:46:17.889606 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:46:17.889780 16271 external_mini_cluster.cc:1428] Setting key f6b2063d7de4b973fdd86d648766e25b
I20260812 14:46:17.891625 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.884037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:47859 (local address 127.15.227.254:36657)
0812 14:46:17.884088 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.884098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.884294 (+ 196us) server_negotiation.cc:407] Connection header received
0812 14:46:17.884697 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.884699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.884708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.884747 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:17.885560 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.886062 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.886984 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.887183 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.887235 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.889082 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:17.889083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:17.889084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:17.889090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:17.890599 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.890958 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.890959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.890959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.891009 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.891180 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.891180 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.891181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.891279 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:17.891323 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.891507 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.891565 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:46:17.891754 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.884015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:36657 (local address 127.15.227.195:47859)
0812 14:46:17.884188 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:17.884222 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:17.884614 (+ 392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:17.884808 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:17.884811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:17.885330 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:17.885495 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.885500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.886135 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.886137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.886906 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.886908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.887242 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.887243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.887332 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.887730 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:17.887737 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:17.888991 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:17.890682 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:17.890690 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:17.890692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:17.890902 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:17.891055 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:17.891056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:17.891056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:17.891140 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:17.891319 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:17.891322 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:17.891418 (+ 96us) client_negotiation.cc:769] Sending connection context
0812 14:46:17.891490 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 14:46:17.891594 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260812 14:46:17.892122 21449 heartbeater.cc:347] Connected to a master server at 127.15.227.254:36657
I20260812 14:46:17.892210 21449 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:17.892340 21449 heartbeater.cc:511] Master 127.15.227.254:36657 requested a full tablet report, sending...
I20260812 14:46:17.892637 20576 ts_manager.cc:194] Registered new tserver with Master: de0da8dd429846e3a26bab8dbf3789a9 (127.15.227.195:42441)
I20260812 14:46:17.892997 20576 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.227.195@KRBTEST.COM'} at 127.15.227.195:47859
I20260812 14:46:17.901266 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:46:17.907411 20653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.902520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53544 (local address 127.15.227.254:36657)
0812 14:46:17.902570 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.902571 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.902669 (+ 98us) server_negotiation.cc:407] Connection header received
0812 14:46:17.902712 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.902713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.902722 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.902763 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:17.903226 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.903682 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.904354 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.904521 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.904560 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.905508 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:17.905509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:17.905509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:17.905513 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:17.906604 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.906885 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.906885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.906885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.906925 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:17.907051 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:17.907052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:17.907052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:17.907161 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:17.907235 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.907294 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.907347 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:46:17.908818 20576 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:46:17.910475 20576 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:46:17.916251 21466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.912270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:37915 (local address 127.0.0.1:60152)
0812 14:46:17.912412 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:17.912445 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:17.912504 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:17.912840 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:17.912843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:17.912845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:17.912923 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.912925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.914230 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.914230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.915225 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.915228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.916025 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.916026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.916100 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.916110 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 14:46:17.916159 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:46:17.916193 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":72,"thread_start_us":47,"threads_started":1}
I20260812 14:46:17.916446 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.912507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.15.227.193:37915)
0812 14:46:17.912698 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.912703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.912710 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:17.912744 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.912745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.912757 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.912822 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:17.913007 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.914130 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.915283 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.915936 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.916027 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.916088 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.916324 (+ 236us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.916368 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":83,"threads_started":1}
I20260812 14:46:17.916695 21469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.912282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:35633 (local address 127.0.0.1:55274)
0812 14:46:17.912626 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:17.912636 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:17.912695 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:17.912854 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:17.912855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:17.912857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:17.912936 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.912938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.914383 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.914384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.915518 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.915520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.916480 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.916480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.916542 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.916546 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:17.916595 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:46:17.916632 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":20,"thread_start_us":48,"threads_started":1}
I20260812 14:46:17.917001 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.912489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55274 (local address 127.15.227.194:35633)
0812 14:46:17.912632 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.912636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.912689 (+ 53us) server_negotiation.cc:407] Connection header received
0812 14:46:17.912756 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.912758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.912773 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.912837 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:17.913016 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.914125 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.915617 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.916327 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.916390 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.916436 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.916884 (+ 448us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.916927 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260812 14:46:17.917112 20818 tablet_service.cc:1467] Processing CreateTablet for tablet b00daf1acc2a48adac2095617eaa8771 (DEFAULT_TABLE table=test-table [id=a0431664a7ca440e9bd7d37271105ff0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:17.917444 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.912270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:42441 (local address 127.0.0.1:51458)
0812 14:46:17.912585 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:17.912594 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:17.912626 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:17.913305 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:17.913306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:17.913307 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:17.913344 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.913346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.914162 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.914163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.915517 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:17.915520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.917222 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.917223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:17.917282 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.917286 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:17.917340 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:46:17.917378 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":72,"thread_start_us":43,"threads_started":1}
I20260812 14:46:17.917475 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:17.912325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.15.227.195:42441)
0812 14:46:17.912942 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:17.912946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:17.912954 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:17.912982 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:17.912983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:17.912996 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:17.913239 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:17.913398 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.914066 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.916471 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:17.917067 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:17.917134 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:17.917188 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:17.917369 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 14:46:17.917408 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":76,"threads_started":1}
I20260812 14:46:17.917484 20818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00daf1acc2a48adac2095617eaa8771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:17.917662 21066 tablet_service.cc:1467] Processing CreateTablet for tablet b00daf1acc2a48adac2095617eaa8771 (DEFAULT_TABLE table=test-table [id=a0431664a7ca440e9bd7d37271105ff0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:17.917915 21066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00daf1acc2a48adac2095617eaa8771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:17.918074 21310 tablet_service.cc:1467] Processing CreateTablet for tablet b00daf1acc2a48adac2095617eaa8771 (DEFAULT_TABLE table=test-table [id=a0431664a7ca440e9bd7d37271105ff0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:17.918370 21310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b00daf1acc2a48adac2095617eaa8771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:17.920624 21475 tablet_bootstrap.cc:492] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Bootstrap starting.
I20260812 14:46:17.921037 21476 tablet_bootstrap.cc:492] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569: Bootstrap starting.
I20260812 14:46:17.921384 21478 tablet_bootstrap.cc:492] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Bootstrap starting.
I20260812 14:46:17.921425 21475 tablet_bootstrap.cc:654] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:17.921669 21476 tablet_bootstrap.cc:654] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:17.922077 21478 tablet_bootstrap.cc:654] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:17.922089 21475 log.cc:824] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:17.922169 21476 log.cc:824] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:17.922582 21478 log.cc:824] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:17.922851 21475 tablet_bootstrap.cc:492] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: No bootstrap required, opened a new log
I20260812 14:46:17.922927 21475 ts_tablet_manager.cc:1397] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260812 14:46:17.922993 21476 tablet_bootstrap.cc:492] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569: No bootstrap required, opened a new log
I20260812 14:46:17.923056 21476 ts_tablet_manager.cc:1397] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:46:17.923259 21478 tablet_bootstrap.cc:492] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: No bootstrap required, opened a new log
I20260812 14:46:17.923318 21478 ts_tablet_manager.cc:1397] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:46:17.924710 21478 raft_consensus.cc:348] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:17.924844 21478 raft_consensus.cc:374] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:17.924878 21478 raft_consensus.cc:729] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de0da8dd429846e3a26bab8dbf3789a9, State: Initialized, Role: FOLLOWER
I20260812 14:46:17.924798 21476 raft_consensus.cc:348] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:17.924788 21475 raft_consensus.cc:348] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:17.924949 21476 raft_consensus.cc:374] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:17.924954 21475 raft_consensus.cc:374] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:17.924993 21475 raft_consensus.cc:729] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c598f39887254b5d8da360b1bf3cd0a0, State: Initialized, Role: FOLLOWER
I20260812 14:46:17.924993 21476 raft_consensus.cc:729] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6473ba4c058421dbb8e7c20bcddb569, State: Initialized, Role: FOLLOWER
I20260812 14:46:17.925004 21478 consensus_queue.cc:260] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:17.925119 21476 consensus_queue.cc:260] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:17.925119 21475 consensus_queue.cc:260] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:17.925300 21478 ts_tablet_manager.cc:1428] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 14:46:17.925396 21476 ts_tablet_manager.cc:1428] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:46:17.925396 21475 ts_tablet_manager.cc:1428] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:46:17.925419 21449 heartbeater.cc:503] Master 127.15.227.254:36657 was elected leader, sending a full tablet report...
I20260812 14:46:17.925513 21198 heartbeater.cc:503] Master 127.15.227.254:36657 was elected leader, sending a full tablet report...
I20260812 14:46:17.925567 20940 heartbeater.cc:503] Master 127.15.227.254:36657 was elected leader, sending a full tablet report...
W20260812 14:46:17.990839 21203 tablet.cc:2369] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:46:18.028378 21489 raft_consensus.cc:482] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:18.028486 21489 raft_consensus.cc:504] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:18.028787 21489 leader_election.cc:288] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6473ba4c058421dbb8e7c20bcddb569 (127.15.227.193:37915), c598f39887254b5d8da360b1bf3cd0a0 (127.15.227.194:35633)
I20260812 14:46:18.032512 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.028935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:37915 (local address 127.15.227.195:56993)
0812 14:46:18.028991 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.029003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.029044 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.029206 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.029207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.029210 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:18.029282 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.029284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.030218 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.030220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.031475 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.031478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.032242 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.032243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.032328 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.032333 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.032388 (+ 55us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.032434 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":42}
I20260812 14:46:18.032560 21471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.029031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:49487 (local address 127.15.227.194:35633)
0812 14:46:18.029077 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.029078 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.029226 (+ 148us) server_negotiation.cc:407] Connection header received
0812 14:46:18.029253 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.029254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.029261 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.029301 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:18.029422 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.030122 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.031560 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.032143 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.032230 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.032237 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.032449 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.032493 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:46:18.032582 21474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.029036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:56993 (local address 127.15.227.193:37915)
0812 14:46:18.029084 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.029085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.029094 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:46:18.029122 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.029123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.029133 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.029181 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:18.029351 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.030114 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.031560 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.032150 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.032218 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.032228 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.032417 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.032495 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:46:18.032809 21495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.028952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:35633 (local address 127.15.227.195:49487)
0812 14:46:18.029147 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.029156 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.029203 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.029304 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.029305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.029306 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:18.029364 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.029365 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.030219 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.030220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.031475 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.031478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.032282 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.032284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.032359 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.032363 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.032671 (+ 308us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.032730 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":29,"thread_start_us":105,"threads_started":1}
I20260812 14:46:18.033191 21104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00daf1acc2a48adac2095617eaa8771" candidate_uuid: "de0da8dd429846e3a26bab8dbf3789a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c598f39887254b5d8da360b1bf3cd0a0" is_pre_election: true
I20260812 14:46:18.033232 20855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00daf1acc2a48adac2095617eaa8771" candidate_uuid: "de0da8dd429846e3a26bab8dbf3789a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6473ba4c058421dbb8e7c20bcddb569" is_pre_election: true
I20260812 14:46:18.033367 21104 raft_consensus.cc:2454] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de0da8dd429846e3a26bab8dbf3789a9 in term 0.
I20260812 14:46:18.033366 20855 raft_consensus.cc:2454] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de0da8dd429846e3a26bab8dbf3789a9 in term 0.
I20260812 14:46:18.033608 21234 leader_election.cc:302] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c598f39887254b5d8da360b1bf3cd0a0, de0da8dd429846e3a26bab8dbf3789a9; no voters:
I20260812 14:46:18.033753 21489 raft_consensus.cc:2781] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:46:18.033811 21489 raft_consensus.cc:482] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:46:18.033833 21489 raft_consensus.cc:3037] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:18.034585 21489 raft_consensus.cc:504] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:18.034714 21489 leader_election.cc:288] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [CANDIDATE]: Term 1 election: Requested vote from peers d6473ba4c058421dbb8e7c20bcddb569 (127.15.227.193:37915), c598f39887254b5d8da360b1bf3cd0a0 (127.15.227.194:35633)
I20260812 14:46:18.034878 20855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00daf1acc2a48adac2095617eaa8771" candidate_uuid: "de0da8dd429846e3a26bab8dbf3789a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6473ba4c058421dbb8e7c20bcddb569"
I20260812 14:46:18.034906 21104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b00daf1acc2a48adac2095617eaa8771" candidate_uuid: "de0da8dd429846e3a26bab8dbf3789a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c598f39887254b5d8da360b1bf3cd0a0"
I20260812 14:46:18.034973 21104 raft_consensus.cc:3037] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:18.034973 20855 raft_consensus.cc:3037] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:18.035904 20855 raft_consensus.cc:2454] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de0da8dd429846e3a26bab8dbf3789a9 in term 1.
I20260812 14:46:18.035904 21104 raft_consensus.cc:2454] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de0da8dd429846e3a26bab8dbf3789a9 in term 1.
I20260812 14:46:18.036087 21236 leader_election.cc:302] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6473ba4c058421dbb8e7c20bcddb569, de0da8dd429846e3a26bab8dbf3789a9; no voters:
I20260812 14:46:18.036178 21489 raft_consensus.cc:2781] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:18.036345 21489 raft_consensus.cc:686] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 1 LEADER]: Becoming Leader. State: Replica: de0da8dd429846e3a26bab8dbf3789a9, State: Running, Role: LEADER
I20260812 14:46:18.036435 21489 consensus_queue.cc:237] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } }
I20260812 14:46:18.037297 20576 catalog_manager.cc:5495] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to de0da8dd429846e3a26bab8dbf3789a9 (127.15.227.195). New cstate: current_term: 1 leader_uuid: "de0da8dd429846e3a26bab8dbf3789a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de0da8dd429846e3a26bab8dbf3789a9" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 42441 } health_report { overall_health: HEALTHY } } }
I20260812 14:46:18.058883 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.056859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51462 (local address 127.15.227.195:42441)
0812 14:46:18.056909 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.056911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.056936 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:46:18.056960 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.056961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.056975 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.057022 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:18.057125 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.057612 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.058332 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.058507 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.058550 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.058668 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:18.058751 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:18.058802 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.058820 (+ 18us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.058835 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:46:18.060464 21104 raft_consensus.cc:1261] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 1 FOLLOWER]: Refusing update from remote peer de0da8dd429846e3a26bab8dbf3789a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:18.060472 20855 raft_consensus.cc:1261] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569 [term 1 FOLLOWER]: Refusing update from remote peer de0da8dd429846e3a26bab8dbf3789a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:18.060807 21496 consensus_queue.cc:1035] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c598f39887254b5d8da360b1bf3cd0a0" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 35633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:18.060918 21489 consensus_queue.cc:1035] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6473ba4c058421dbb8e7c20bcddb569" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 37915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:18.063395 21507 mvcc.cc:205] Tried to move back new op lower bound from 7317692326133420032 to 7317692326037876736. Current Snapshot: MvccSnapshot[applied={T|T < 7317692326133420032}]
I20260812 14:46:18.063925 21504 mvcc.cc:205] Tried to move back new op lower bound from 7317692326133420032 to 7317692326037876736. Current Snapshot: MvccSnapshot[applied={T|T < 7317692326133420032 or (T in {7317692326133420032})}]
I20260812 14:46:18.065461 20576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:18.065552 20576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:18.066265 20576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bcf76afe12e4be98dc7328132586d44: Sending DeleteTablet for 3 replicas of tablet b00daf1acc2a48adac2095617eaa8771
I20260812 14:46:18.066588 21310 tablet_service.cc:1514] Processing DeleteTablet for tablet b00daf1acc2a48adac2095617eaa8771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:18 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:51458
I20260812 14:46:18.066596 20818 tablet_service.cc:1514] Processing DeleteTablet for tablet b00daf1acc2a48adac2095617eaa8771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:18 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:60152
I20260812 14:46:18.066586 21066 tablet_service.cc:1514] Processing DeleteTablet for tablet b00daf1acc2a48adac2095617eaa8771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:18 UTC) from {username='kudu', principal='kudu/127.15.227.254@KRBTEST.COM'} at 127.0.0.1:55274
I20260812 14:46:18.066855 21514 tablet_replica.cc:330] T b00daf1acc2a48adac2095617eaa8771 P d6473ba4c058421dbb8e7c20bcddb569: stopping tablet replica
I20260812 14:46:18.066924 21518 tablet_replica.cc:330] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: stopping tablet replica
I20260812 14:46:18.067013 21518 raft_consensus.cc:2229] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 14:46:18.067076 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 20701
I20260812 14:46:18.067090 21518 raft_consensus.cc:2258] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:18.120335 21518 ts_tablet_manager.cc:1905] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:18.120379 21515 tablet_replica.cc:330] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: stopping tablet replica
I20260812 14:46:18.120486 21515 raft_consensus.cc:2229] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:18.120602 21515 raft_consensus.cc:2258] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:18.122001 21518 ts_tablet_manager.cc:1918] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:18.122128 21518 log.cc:1196] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/b00daf1acc2a48adac2095617eaa8771
I20260812 14:46:18.122458 21515 ts_tablet_manager.cc:1905] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:18.122502 21518 ts_tablet_manager.cc:1939] T b00daf1acc2a48adac2095617eaa8771 P de0da8dd429846e3a26bab8dbf3789a9: Deleting consensus metadata
I20260812 14:46:18.122908 20542 catalog_manager.cc:4841] TS de0da8dd429846e3a26bab8dbf3789a9 (127.15.227.195:42441): tablet b00daf1acc2a48adac2095617eaa8771 (table test-table [id=a0431664a7ca440e9bd7d37271105ff0]) successfully deleted
I20260812 14:46:18.124202 21515 ts_tablet_manager.cc:1918] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:18.124281 21515 log.cc:1196] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/b00daf1acc2a48adac2095617eaa8771
I20260812 14:46:18.124537 21515 ts_tablet_manager.cc:1939] T b00daf1acc2a48adac2095617eaa8771 P c598f39887254b5d8da360b1bf3cd0a0: Deleting consensus metadata
I20260812 14:46:18.124958 20534 catalog_manager.cc:4841] TS c598f39887254b5d8da360b1bf3cd0a0 (127.15.227.194:35633): tablet b00daf1acc2a48adac2095617eaa8771 (table test-table [id=a0431664a7ca440e9bd7d37271105ff0]) successfully deleted
W20260812 14:46:18.125444 20541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:37915 (error 108)
I20260812 14:46:18.125679 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 20949
I20260812 14:46:18.136307 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21208
I20260812 14:46:18.136458 21468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.136323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:37915 (local address 127.0.0.1:60168)
0812 14:46:18.136380 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.136423 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:37915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 14:46:18.143996 20541 catalog_manager.cc:4584] TS d6473ba4c058421dbb8e7c20bcddb569 (127.15.227.193:37915): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b00daf1acc2a48adac2095617eaa8771: Network error: Client connection negotiation failed: client connection to 127.15.227.193:37915: connect: Connection refused (error 111)
I20260812 14:46:18.147418 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 20508
2026-08-12T14:46:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2151 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T14:46:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:18Z Disabled control of system clock
I20260812 14:46:18.179914 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:38043
--webserver_interface=127.15.227.254
--webserver_port=0
--builtin_ntp_servers=127.15.227.212:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:38043
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:18.258656 21524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:18.258855 21524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:18.258875 21524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:18.259987 21524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:18.260027 21524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:18.260039 21524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:18.260051 21524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:18.260061 21524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:18.261377 21524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:38043
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:38043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:18.261595 21524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:18.261854 21524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:18.266358 21534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:18.266309 21531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:18.266326 21532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:18.266454 21524 server_base.cc:1034] running on GCE node
I20260812 14:46:18.266839 21524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:18.267112 21524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:18.268270 21524 hybrid_clock.cc:648] HybridClock initialized: now 1786545978268254 us; error 29 us; skew 500 ppm
I20260812 14:46:18.269447 21524 webserver.cc:460] Webserver started at http://127.15.227.254:40005/ using document root <none> and password file <none>
I20260812 14:46:18.269675 21524 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:18.269724 21524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:18.269858 21524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:18.270870 21524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "efa45c90d9554f44a607d8251a1cfca6"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "1ceb7706732c9716257ba200783fd2e3"
server_key_iv: "18fc5d5773467fa0e31c43082003fe20"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.271193 21524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "efa45c90d9554f44a607d8251a1cfca6"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "1ceb7706732c9716257ba200783fd2e3"
server_key_iv: "18fc5d5773467fa0e31c43082003fe20"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.272444 21524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:18.273097 21546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.273247 21524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:18.273326 21524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "efa45c90d9554f44a607d8251a1cfca6"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "1ceb7706732c9716257ba200783fd2e3"
server_key_iv: "18fc5d5773467fa0e31c43082003fe20"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.273392 21524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:18.288018 21524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:18.288251 21524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:18.288371 21524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:18.291993 21524 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:38043
I20260812 14:46:18.292023 21654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:38043 every 8 connection(s)
I20260812 14:46:18.292299 21524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:18.293202 21656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:18.293848 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21524
I20260812 14:46:18.293963 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:18.294196 16271 external_mini_cluster.cc:1428] Setting key 36c15d2c5906bd3c0f51882a5215f8c9
I20260812 14:46:18.296018 21656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6: Bootstrap starting.
I20260812 14:46:18.296587 21656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:18.296904 21656 log.cc:824] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:18.297611 21656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6: No bootstrap required, opened a new log
I20260812 14:46:18.299235 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.294686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55848 (local address 127.15.227.254:38043)
0812 14:46:18.294861 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.294868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.294891 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:46:18.295274 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.295280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.295334 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.295436 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:18.295665 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.296504 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.297492 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.297899 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.297980 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.298641 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:18.298645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:18.298645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:18.298793 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:18.298824 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:18.298834 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:18.298891 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.298985 (+ 94us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.299046 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260812 14:46:18.299697 21656 raft_consensus.cc:348] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa45c90d9554f44a607d8251a1cfca6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38043 } }
I20260812 14:46:18.299831 21656 raft_consensus.cc:374] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:18.299867 21656 raft_consensus.cc:729] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efa45c90d9554f44a607d8251a1cfca6, State: Initialized, Role: FOLLOWER
I20260812 14:46:18.299989 21656 consensus_queue.cc:260] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa45c90d9554f44a607d8251a1cfca6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38043 } }
I20260812 14:46:18.300063 21656 raft_consensus.cc:388] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:18.300084 21656 raft_consensus.cc:482] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:18.300119 21656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:18.300577 21656 raft_consensus.cc:504] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa45c90d9554f44a607d8251a1cfca6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38043 } }
I20260812 14:46:18.300669 21656 leader_election.cc:302] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efa45c90d9554f44a607d8251a1cfca6; no voters:
I20260812 14:46:18.300794 21656 leader_election.cc:288] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:18.300843 21665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:18.300997 21665 raft_consensus.cc:686] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [term 1 LEADER]: Becoming Leader. State: Replica: efa45c90d9554f44a607d8251a1cfca6, State: Running, Role: LEADER
I20260812 14:46:18.301025 21656 sys_catalog.cc:564] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:18.301093 21665 consensus_queue.cc:237] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa45c90d9554f44a607d8251a1cfca6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38043 } }
I20260812 14:46:18.301469 21672 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efa45c90d9554f44a607d8251a1cfca6. Latest consensus state: current_term: 1 leader_uuid: "efa45c90d9554f44a607d8251a1cfca6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa45c90d9554f44a607d8251a1cfca6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38043 } } }
I20260812 14:46:18.301560 21672 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:18.301707 21669 sys_catalog.cc:455] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efa45c90d9554f44a607d8251a1cfca6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efa45c90d9554f44a607d8251a1cfca6" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 38043 } } }
I20260812 14:46:18.301755 21682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:18.301774 21669 sys_catalog.cc:458] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:18.302428 21682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:18.303855 21682 catalog_manager.cc:1347] Generated new cluster ID: 66a4e68959e94b6d89a277a1320822b9
I20260812 14:46:18.303908 21682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:18.319597 21682 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:18.320259 21682 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:18.330534 21682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6: Generated new TSK 0
I20260812 14:46:18.330685 21682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:18.333489 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.193:0
--local_ip_for_outbound_sockets=127.15.227.193
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38043
--builtin_ntp_servers=127.15.227.212:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:18.413499 21698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:18.413677 21698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:18.413707 21698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:18.414841 21698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:18.414880 21698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:18.414913 21698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.193
I20260812 14:46:18.416213 21698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.227.193
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.193
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:18.416432 21698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:18.416692 21698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:18.421332 21706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:18.421366 21707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:18.421636 21698 server_base.cc:1034] running on GCE node
W20260812 14:46:18.421386 21709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:18.421851 21698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:18.422093 21698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:18.423240 21698 hybrid_clock.cc:648] HybridClock initialized: now 1786545978423222 us; error 40 us; skew 500 ppm
I20260812 14:46:18.424388 21698 webserver.cc:460] Webserver started at http://127.15.227.193:33665/ using document root <none> and password file <none>
I20260812 14:46:18.424611 21698 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:18.424661 21698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:18.424788 21698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:18.425722 21698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data/instance:
uuid: "1276de671b874ec6b67524983295f870"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "93e76370e739cf3552b6747f8d689250"
server_key_iv: "90482585ab23d1dfe4f912fc6f6263e2"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.426064 21698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance:
uuid: "1276de671b874ec6b67524983295f870"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "93e76370e739cf3552b6747f8d689250"
server_key_iv: "90482585ab23d1dfe4f912fc6f6263e2"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.427263 21698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:46:18.427930 21717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.428109 21698 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:18.428211 21698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/wal
uuid: "1276de671b874ec6b67524983295f870"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "93e76370e739cf3552b6747f8d689250"
server_key_iv: "90482585ab23d1dfe4f912fc6f6263e2"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.428359 21698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:18.459122 21698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:18.459395 21698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:18.459514 21698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:18.459712 21698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:18.460014 21698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:18.460048 21698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.460069 21698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:18.460080 21698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.465402 21698 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.193:39597
I20260812 14:46:18.465476 21948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.193:39597 every 8 connection(s)
I20260812 14:46:18.465740 21698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/data/info.pb
I20260812 14:46:18.468930 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21698
I20260812 14:46:18.469061 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-0/wal/instance
I20260812 14:46:18.469267 16271 external_mini_cluster.cc:1428] Setting key b9cd495acd13e51f789c5e55a742b87a
I20260812 14:46:18.470283 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.194:0
--local_ip_for_outbound_sockets=127.15.227.194
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38043
--builtin_ntp_servers=127.15.227.212:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 14:46:18.470912 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.466443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:48601 (local address 127.15.227.254:38043)
0812 14:46:18.466496 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.466498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.466728 (+ 230us) server_negotiation.cc:407] Connection header received
0812 14:46:18.467227 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.467229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.467244 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.467298 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:18.467926 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.468421 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.469431 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.469642 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.469707 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.470667 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:18.470669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:18.470669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:18.470674 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:18.470691 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:18.470696 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:18.470745 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.470807 (+ 62us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.470838 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260812 14:46:18.471066 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.466430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:38043 (local address 127.15.227.193:48601)
0812 14:46:18.466617 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.466650 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.467123 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.467359 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.467362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.467370 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:18.467841 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.467846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.468504 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.468504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.469356 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.469360 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.469733 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.469735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.469883 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.470510 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:18.470510 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:18.470546 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:18.470570 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:18.470748 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:18.470751 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:18.470753 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.470798 (+ 45us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.470874 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260812 14:46:18.472199 21953 heartbeater.cc:347] Connected to a master server at 127.15.227.254:38043
I20260812 14:46:18.472302 21953 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:18.472493 21953 heartbeater.cc:511] Master 127.15.227.254:38043 requested a full tablet report, sending...
I20260812 14:46:18.473026 21578 ts_manager.cc:194] Registered new tserver with Master: 1276de671b874ec6b67524983295f870 (127.15.227.193:39597)
I20260812 14:46:18.473639 21578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.193:48601
W20260812 14:46:18.549858 21960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:18.550055 21960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:18.550087 21960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:18.551204 21960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:18.551246 21960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:18.551278 21960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.194
I20260812 14:46:18.552577 21960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.227.194
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.194
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:18.552799 21960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:18.553050 21960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:18.557669 21970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:18.557704 21969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:18.557755 21976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:18.557770 21960 server_base.cc:1034] running on GCE node
I20260812 14:46:18.558096 21960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:18.558348 21960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:18.559485 21960 hybrid_clock.cc:648] HybridClock initialized: now 1786545978559468 us; error 31 us; skew 500 ppm
I20260812 14:46:18.560559 21960 webserver.cc:460] Webserver started at http://127.15.227.194:41643/ using document root <none> and password file <none>
I20260812 14:46:18.560760 21960 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:18.560807 21960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:18.560928 21960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:18.561885 21960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data/instance:
uuid: "249ef7e08e644a5c832ca2913494a580"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "60599851e3d041f435ac234d010b1829"
server_key_iv: "2a9ed659f645794fd230bf1d9952149c"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.562230 21960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance:
uuid: "249ef7e08e644a5c832ca2913494a580"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "60599851e3d041f435ac234d010b1829"
server_key_iv: "2a9ed659f645794fd230bf1d9952149c"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.563375 21960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:46:18.563997 21990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.564141 21960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:18.564231 21960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal
uuid: "249ef7e08e644a5c832ca2913494a580"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "60599851e3d041f435ac234d010b1829"
server_key_iv: "2a9ed659f645794fd230bf1d9952149c"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.564296 21960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:18.574476 21960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:18.574697 21960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:18.574822 21960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:18.575001 21960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:18.575289 21960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:18.575323 21960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.575348 21960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:18.575371 21960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.580717 21960 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.194:41283
I20260812 14:46:18.580754 22221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.194:41283 every 8 connection(s)
I20260812 14:46:18.581013 21960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/data/info.pb
I20260812 14:46:18.585238 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.581663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.194:48569 (local address 127.15.227.254:38043)
0812 14:46:18.581719 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.581721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.581900 (+ 179us) server_negotiation.cc:407] Connection header received
0812 14:46:18.582320 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.582321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.582330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.582381 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:18.582929 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.583401 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.584245 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.584417 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.584458 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.585040 (+ 582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:18.585041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:18.585042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:18.585046 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:18.585059 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:18.585063 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:18.585102 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.585151 (+ 49us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.585177 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:46:18.585395 22226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.581630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:38043 (local address 127.15.227.194:48569)
0812 14:46:18.581798 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.581832 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.582251 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.582425 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.582428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.582435 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:18.582869 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.582872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.583461 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.583462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.584177 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.584179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.584467 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.584467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.584559 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.584930 (+ 371us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:18.584931 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:18.584954 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:18.584973 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:18.585102 (+ 129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:18.585104 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:18.585106 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.585146 (+ 40us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.585211 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260812 14:46:18.585749 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 21960
I20260812 14:46:18.585851 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal/instance
I20260812 14:46:18.585826 22223 heartbeater.cc:347] Connected to a master server at 127.15.227.254:38043
I20260812 14:46:18.585916 22223 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:18.586086 16271 external_mini_cluster.cc:1428] Setting key 4a73b27bc9fa6bde1f8609672b213203
I20260812 14:46:18.586092 22223 heartbeater.cc:511] Master 127.15.227.254:38043 requested a full tablet report, sending...
I20260812 14:46:18.586478 21577 ts_manager.cc:194] Registered new tserver with Master: 249ef7e08e644a5c832ca2913494a580 (127.15.227.194:41283)
I20260812 14:46:18.586910 21577 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.194:48569
I20260812 14:46:18.586984 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.227.195:0
--local_ip_for_outbound_sockets=127.15.227.195
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38043
--builtin_ntp_servers=127.15.227.212:35207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 14:46:18.665308 22228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:18.665485 22228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:18.665516 22228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:18.666671 22228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:18.666712 22228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:18.666754 22228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.227.195
I20260812 14:46:18.668097 22228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.227.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.227.195
--webserver_port=0
--tserver_master_addrs=127.15.227.254:38043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.227.195
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:18.668318 22228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:18.668576 22228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:18.673264 22236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:18.673262 22235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:18.673243 22247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:18.673413 22228 server_base.cc:1034] running on GCE node
I20260812 14:46:18.673636 22228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:18.673864 22228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:18.674991 22228 hybrid_clock.cc:648] HybridClock initialized: now 1786545978674972 us; error 37 us; skew 500 ppm
I20260812 14:46:18.676136 22228 webserver.cc:460] Webserver started at http://127.15.227.195:40061/ using document root <none> and password file <none>
I20260812 14:46:18.676389 22228 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:18.676450 22228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:18.676576 22228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:18.677564 22228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data/instance:
uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "cc56dd60db940c240b8717e7287e2899"
server_key_iv: "a27985d12c7489b3467ad11277c08b45"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.677881 22228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance:
uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "cc56dd60db940c240b8717e7287e2899"
server_key_iv: "a27985d12c7489b3467ad11277c08b45"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.679077 22228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:46:18.679742 22266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.679893 22228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:18.679975 22228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal
uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f"
format_stamp: "Formatted at 2026-08-12 14:46:18 on dist-test-slave-q9h9"
server_key: "cc56dd60db940c240b8717e7287e2899"
server_key_iv: "a27985d12c7489b3467ad11277c08b45"
server_key_version: "encryptionkey@0"
I20260812 14:46:18.680042 22228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:18.695969 22228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:18.696206 22228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:18.696332 22228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:18.696514 22228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 14:46:18.696795 22228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 14:46:18.696828 22228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.696853 22228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 14:46:18.696869 22228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:18.702315 22228 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.195:39487
I20260812 14:46:18.702351 22482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.195:39487 every 8 connection(s)
I20260812 14:46:18.702637 22228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/data/info.pb
I20260812 14:46:18.707090 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.703262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.195:55071 (local address 127.15.227.254:38043)
0812 14:46:18.703310 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.703311 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.703514 (+ 203us) server_negotiation.cc:407] Connection header received
0812 14:46:18.703932 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.703933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.703942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.703993 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:18.704572 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.705125 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.706038 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.706237 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.706286 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.706901 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:18.706902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:18.706902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:18.706906 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:18.706921 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:18.706925 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:18.706963 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.707013 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.707033 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:46:18.707273 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.703235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:38043 (local address 127.15.227.195:55071)
0812 14:46:18.703404 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.703440 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.703861 (+ 421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.704040 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.704043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.704051 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:18.704512 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.704516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.705194 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.705195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.705962 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.705964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.706292 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.706293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.706389 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.706787 (+ 398us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 14:46:18.706787 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:18.706812 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 14:46:18.706832 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:18.706964 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:18.706966 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:18.706968 (+ 2us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.707009 (+ 41us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.707077 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260812 14:46:18.707695 22483 heartbeater.cc:347] Connected to a master server at 127.15.227.254:38043
I20260812 14:46:18.707767 22483 heartbeater.cc:464] Registering TS with master...
I20260812 14:46:18.707922 22483 heartbeater.cc:511] Master 127.15.227.254:38043 requested a full tablet report, sending...
I20260812 14:46:18.708250 21577 ts_manager.cc:194] Registered new tserver with Master: d641e52ba76d4e06a79d2b99e3a0dd1f (127.15.227.195:39487)
I20260812 14:46:18.708609 21577 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.227.195:55071
I20260812 14:46:18.711783 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 22228
I20260812 14:46:18.711874 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal/instance
I20260812 14:46:18.712018 16271 external_mini_cluster.cc:1428] Setting key e67cf74af1be260e21ad3dcd025402b3
I20260812 14:46:18.712774 16271 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 14:46:18.716548 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.714005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.15.227.254:38043)
0812 14:46:18.714079 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.714081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.714163 (+ 82us) server_negotiation.cc:407] Connection header received
0812 14:46:18.714211 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.714212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.714222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.714266 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:18.714412 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.714866 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.715588 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.715787 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.715839 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.716378 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:18.716379 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:18.716379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:18.716383 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:18.716394 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:18.716397 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:18.716439 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.716486 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.716503 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 14:46:18.717918 21577 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 14:46:18.719422 21577 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 14:46:18.725168 22503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.721342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:39597 (local address 127.0.0.1:42832)
0812 14:46:18.721666 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.721674 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.721706 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.721921 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.721924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.721927 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:18.722006 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.722008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.722915 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.722917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.723975 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.723978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.724890 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.724891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.724972 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.724987 (+ 15us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.725049 (+ 62us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.725100 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":98,"threads_started":1}
I20260812 14:46:18.725363 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.721331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:39487 (local address 127.0.0.1:35474)
0812 14:46:18.721488 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.721523 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.721581 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.722091 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.722091 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.722093 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:18.722122 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.722122 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.723375 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.723376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.724376 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.724378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.725033 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.725034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.725098 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.725102 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.725156 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.725196 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":48,"threads_started":1}
I20260812 14:46:18.725410 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.721575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.15.227.193:39597)
0812 14:46:18.721766 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.721770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.721778 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:18.721809 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.721811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.721822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.721894 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:18.722122 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.722838 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.724069 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.724802 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.724877 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.724956 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.725294 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.725334 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260812 14:46:18.725621 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.721369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35474 (local address 127.15.227.195:39487)
0812 14:46:18.721926 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.721930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.721937 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:18.721965 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.721966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.721978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.722186 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:18.722227 (+ 41us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.723294 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.724445 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.724904 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.725003 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.725084 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.725173 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.725207 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":81,"threads_started":1}
I20260812 14:46:18.726073 21825 tablet_service.cc:1467] Processing CreateTablet for tablet 64bc434f9c3f4a96945cc19d3e12b695 (DEFAULT_TABLE table=test-table [id=60bc66a3bca348469cba2aba9372e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:18.726449 21825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bc434f9c3f4a96945cc19d3e12b695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:18.726425 22370 tablet_service.cc:1467] Processing CreateTablet for tablet 64bc434f9c3f4a96945cc19d3e12b695 (DEFAULT_TABLE table=test-table [id=60bc66a3bca348469cba2aba9372e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:18.726652 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.721342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:41283 (local address 127.0.0.1:55796)
0812 14:46:18.721783 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.721790 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.721829 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.722070 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.722071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.722073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:18.722115 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.722117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.724465 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.724466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.725448 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.725451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.726437 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.726438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.726494 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.726499 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.726553 (+ 54us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.726590 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":7,"thread_start_us":82,"threads_started":1}
I20260812 14:46:18.726706 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.721567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55796 (local address 127.15.227.194:41283)
0812 14:46:18.721737 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.721742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.721837 (+ 95us) server_negotiation.cc:407] Connection header received
0812 14:46:18.721883 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.721885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.721900 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.721970 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:18.722194 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.723290 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.725537 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.726256 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.726323 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.726400 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.726585 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.726628 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260812 14:46:18.726734 22370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bc434f9c3f4a96945cc19d3e12b695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:18.727341 22085 tablet_service.cc:1467] Processing CreateTablet for tablet 64bc434f9c3f4a96945cc19d3e12b695 (DEFAULT_TABLE table=test-table [id=60bc66a3bca348469cba2aba9372e2fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 14:46:18.727662 22085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64bc434f9c3f4a96945cc19d3e12b695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:18.729430 22509 tablet_bootstrap.cc:492] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Bootstrap starting.
I20260812 14:46:18.730050 22509 tablet_bootstrap.cc:654] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:18.730181 22510 tablet_bootstrap.cc:492] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870: Bootstrap starting.
I20260812 14:46:18.730365 22509 log.cc:824] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:18.730836 22510 tablet_bootstrap.cc:654] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:18.731031 22509 tablet_bootstrap.cc:492] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: No bootstrap required, opened a new log
I20260812 14:46:18.731089 22509 ts_tablet_manager.cc:1397] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260812 14:46:18.731123 22510 log.cc:824] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:18.731761 22510 tablet_bootstrap.cc:492] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870: No bootstrap required, opened a new log
I20260812 14:46:18.731815 22510 ts_tablet_manager.cc:1397] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260812 14:46:18.732940 22509 raft_consensus.cc:348] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.733063 22509 raft_consensus.cc:374] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:18.733094 22509 raft_consensus.cc:729] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d641e52ba76d4e06a79d2b99e3a0dd1f, State: Initialized, Role: FOLLOWER
I20260812 14:46:18.733201 22509 consensus_queue.cc:260] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.733451 22509 ts_tablet_manager.cc:1428] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 14:46:18.733489 22483 heartbeater.cc:503] Master 127.15.227.254:38043 was elected leader, sending a full tablet report...
I20260812 14:46:18.730723 22513 tablet_bootstrap.cc:492] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Bootstrap starting.
I20260812 14:46:18.733718 22510 raft_consensus.cc:348] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.733872 22510 raft_consensus.cc:374] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:18.733917 22510 raft_consensus.cc:729] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1276de671b874ec6b67524983295f870, State: Initialized, Role: FOLLOWER
I20260812 14:46:18.734040 22510 consensus_queue.cc:260] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.734295 22510 ts_tablet_manager.cc:1428] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 14:46:18.734371 22513 tablet_bootstrap.cc:654] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:18.734421 21953 heartbeater.cc:503] Master 127.15.227.254:38043 was elected leader, sending a full tablet report...
I20260812 14:46:18.734652 22513 log.cc:824] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:18.735383 22513 tablet_bootstrap.cc:492] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: No bootstrap required, opened a new log
I20260812 14:46:18.735442 22513 ts_tablet_manager.cc:1397] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260812 14:46:18.737069 22513 raft_consensus.cc:348] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.737185 22513 raft_consensus.cc:374] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:18.737205 22513 raft_consensus.cc:729] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 249ef7e08e644a5c832ca2913494a580, State: Initialized, Role: FOLLOWER
I20260812 14:46:18.737282 22513 consensus_queue.cc:260] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.737459 22513 ts_tablet_manager.cc:1428] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 14:46:18.737543 22223 heartbeater.cc:503] Master 127.15.227.254:38043 was elected leader, sending a full tablet report...
I20260812 14:46:18.749251 22519 raft_consensus.cc:482] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 14:46:18.749323 22519 raft_consensus.cc:504] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.749586 22519 leader_election.cc:288] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d641e52ba76d4e06a79d2b99e3a0dd1f (127.15.227.195:39487), 249ef7e08e644a5c832ca2913494a580 (127.15.227.194:41283)
I20260812 14:46:18.752054 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.749758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:59845 (local address 127.15.227.194:41283)
0812 14:46:18.749788 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.749790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.749797 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:18.749855 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.749856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.749865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.749924 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:18.750081 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.750554 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.751426 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.751762 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.751822 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.751828 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.751964 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.751996 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:46:18.752054 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.749791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.227.193:46353 (local address 127.15.227.195:39487)
0812 14:46:18.749829 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.749831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.749919 (+ 88us) server_negotiation.cc:407] Connection header received
0812 14:46:18.749965 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.749966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.749974 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.750045 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 14:46:18.750179 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.750590 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.751440 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.751760 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.751818 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.751824 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.751970 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.751999 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260812 14:46:18.752066 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.749702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.194:41283 (local address 127.15.227.193:59845)
0812 14:46:18.749752 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.749760 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.749796 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.749933 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.749933 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.749935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:18.749991 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.749992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.750621 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.750622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.751347 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.751349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.751832 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.751833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.751904 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.751907 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.751949 (+ 42us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.751988 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":3}
I20260812 14:46:18.752081 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.749704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.195:39487 (local address 127.15.227.193:46353)
0812 14:46:18.749858 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.749868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:18.749920 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:18.750058 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:18.750059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:18.750060 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 14:46:18.750122 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.750124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.750666 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.750667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.751363 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:18.751365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.751837 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.751837 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:18.751909 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.751912 (+ 3us) client_negotiation.cc:769] Sending connection context
0812 14:46:18.751959 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:46:18.751995 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":11,"thread_start_us":81,"threads_started":1}
I20260812 14:46:18.752364 22120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc434f9c3f4a96945cc19d3e12b695" candidate_uuid: "1276de671b874ec6b67524983295f870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "249ef7e08e644a5c832ca2913494a580" is_pre_election: true
I20260812 14:46:18.752383 22404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc434f9c3f4a96945cc19d3e12b695" candidate_uuid: "1276de671b874ec6b67524983295f870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" is_pre_election: true
I20260812 14:46:18.752471 22120 raft_consensus.cc:2454] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1276de671b874ec6b67524983295f870 in term 0.
I20260812 14:46:18.752481 22404 raft_consensus.cc:2454] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1276de671b874ec6b67524983295f870 in term 0.
I20260812 14:46:18.752667 21723 leader_election.cc:302] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1276de671b874ec6b67524983295f870, d641e52ba76d4e06a79d2b99e3a0dd1f; no voters:
I20260812 14:46:18.752796 22519 raft_consensus.cc:2781] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 14:46:18.752866 22519 raft_consensus.cc:482] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 14:46:18.752880 22519 raft_consensus.cc:3037] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:18.753407 22519 raft_consensus.cc:504] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.753515 22519 leader_election.cc:288] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [CANDIDATE]: Term 1 election: Requested vote from peers d641e52ba76d4e06a79d2b99e3a0dd1f (127.15.227.195:39487), 249ef7e08e644a5c832ca2913494a580 (127.15.227.194:41283)
I20260812 14:46:18.753645 22404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc434f9c3f4a96945cc19d3e12b695" candidate_uuid: "1276de671b874ec6b67524983295f870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f"
I20260812 14:46:18.753713 22404 raft_consensus.cc:3037] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:18.753718 22120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64bc434f9c3f4a96945cc19d3e12b695" candidate_uuid: "1276de671b874ec6b67524983295f870" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "249ef7e08e644a5c832ca2913494a580"
I20260812 14:46:18.753790 22120 raft_consensus.cc:3037] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:18.754403 22404 raft_consensus.cc:2454] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1276de671b874ec6b67524983295f870 in term 1.
I20260812 14:46:18.754441 22120 raft_consensus.cc:2454] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1276de671b874ec6b67524983295f870 in term 1.
I20260812 14:46:18.754570 21723 leader_election.cc:302] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1276de671b874ec6b67524983295f870, d641e52ba76d4e06a79d2b99e3a0dd1f; no voters:
I20260812 14:46:18.754676 22519 raft_consensus.cc:2781] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:18.754814 22519 raft_consensus.cc:686] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [term 1 LEADER]: Becoming Leader. State: Replica: 1276de671b874ec6b67524983295f870, State: Running, Role: LEADER
I20260812 14:46:18.754884 22519 consensus_queue.cc:237] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } }
I20260812 14:46:18.755532 21578 catalog_manager.cc:5495] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1276de671b874ec6b67524983295f870 (127.15.227.193). New cstate: current_term: 1 leader_uuid: "1276de671b874ec6b67524983295f870" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1276de671b874ec6b67524983295f870" member_type: VOTER last_known_addr { host: "127.15.227.193" port: 39597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 } health_report { overall_health: UNKNOWN } } }
I20260812 14:46:18.760463 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.758411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.15.227.193:39597)
0812 14:46:18.758471 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:18.758473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:18.758481 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:18.758507 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:18.758508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:18.758524 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:18.758581 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:18.758697 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.759130 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.759808 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:18.759996 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:18.760046 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:18.760192 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:18.760274 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:18.760340 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:18.760377 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 14:46:18.760395 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 14:46:18.761755 22120 raft_consensus.cc:1261] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 1 FOLLOWER]: Refusing update from remote peer 1276de671b874ec6b67524983295f870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:18.761783 22404 raft_consensus.cc:1261] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 1 FOLLOWER]: Refusing update from remote peer 1276de671b874ec6b67524983295f870: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:18.762038 22528 consensus_queue.cc:1035] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "249ef7e08e644a5c832ca2913494a580" member_type: VOTER last_known_addr { host: "127.15.227.194" port: 41283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:18.762152 22519 consensus_queue.cc:1035] T 64bc434f9c3f4a96945cc19d3e12b695 P 1276de671b874ec6b67524983295f870 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d641e52ba76d4e06a79d2b99e3a0dd1f" member_type: VOTER last_known_addr { host: "127.15.227.195" port: 39487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:18.764043 22533 mvcc.cc:205] Tried to move back new op lower bound from 7317692329006325760 to 7317692328980504576. Current Snapshot: MvccSnapshot[applied={T|T < 7317692329006325760}]
I20260812 14:46:18.764503 22534 mvcc.cc:205] Tried to move back new op lower bound from 7317692329006325760 to 7317692328980504576. Current Snapshot: MvccSnapshot[applied={T|T < 7317692329006325760}]
I20260812 14:46:18.764856 22535 mvcc.cc:205] Tried to move back new op lower bound from 7317692329006325760 to 7317692328980504576. Current Snapshot: MvccSnapshot[applied={T|T < 7317692329006325760 or (T in {7317692329006325760})}]
I20260812 14:46:18.766508 21578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:18.766619 21578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 14:46:18.767271 21578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P efa45c90d9554f44a607d8251a1cfca6: Sending DeleteTablet for 3 replicas of tablet 64bc434f9c3f4a96945cc19d3e12b695
I20260812 14:46:18.767573 22085 tablet_service.cc:1514] Processing DeleteTablet for tablet 64bc434f9c3f4a96945cc19d3e12b695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:18 UTC) from {username='slave'} at 127.0.0.1:55796
I20260812 14:46:18.767584 21825 tablet_service.cc:1514] Processing DeleteTablet for tablet 64bc434f9c3f4a96945cc19d3e12b695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:18 UTC) from {username='slave'} at 127.0.0.1:42832
I20260812 14:46:18.767807 22558 tablet_replica.cc:330] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: stopping tablet replica
I20260812 14:46:18.767911 22558 raft_consensus.cc:2229] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:18.767935 22370 tablet_service.cc:1514] Processing DeleteTablet for tablet 64bc434f9c3f4a96945cc19d3e12b695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 14:46:18 UTC) from {username='slave'} at 127.0.0.1:35474
I20260812 14:46:18.768013 22558 raft_consensus.cc:2258] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:18.768296 22560 tablet_replica.cc:330] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: stopping tablet replica
I20260812 14:46:18.768386 22560 raft_consensus.cc:2229] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 14:46:18.768440 22558 ts_tablet_manager.cc:1905] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:18.768458 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21698
I20260812 14:46:18.776007 22560 raft_consensus.cc:2258] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 14:46:18.776988 22558 ts_tablet_manager.cc:1918] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:18.777038 22558 log.cc:1196] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-1/wal/wals/64bc434f9c3f4a96945cc19d3e12b695
I20260812 14:46:18.777194 22558 ts_tablet_manager.cc:1939] T 64bc434f9c3f4a96945cc19d3e12b695 P 249ef7e08e644a5c832ca2913494a580: Deleting consensus metadata
I20260812 14:46:18.777614 22560 ts_tablet_manager.cc:1905] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 14:46:18.777786 21550 catalog_manager.cc:4841] TS 249ef7e08e644a5c832ca2913494a580 (127.15.227.194:41283): tablet 64bc434f9c3f4a96945cc19d3e12b695 (table test-table [id=60bc66a3bca348469cba2aba9372e2fb]) successfully deleted
I20260812 14:46:18.778787 22560 ts_tablet_manager.cc:1918] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 14:46:18.778836 22560 log.cc:1196] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Deleting WAL directory at /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786545740023890-16271-0/minicluster-data/ts-2/wal/wals/64bc434f9c3f4a96945cc19d3e12b695
I20260812 14:46:18.779031 22560 ts_tablet_manager.cc:1939] T 64bc434f9c3f4a96945cc19d3e12b695 P d641e52ba76d4e06a79d2b99e3a0dd1f: Deleting consensus metadata
I20260812 14:46:18.779379 21551 catalog_manager.cc:4841] TS d641e52ba76d4e06a79d2b99e3a0dd1f (127.15.227.195:39487): tablet 64bc434f9c3f4a96945cc19d3e12b695 (table test-table [id=60bc66a3bca348469cba2aba9372e2fb]) successfully deleted
W20260812 14:46:18.783974 21548 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.227.193:39597 (error 108)
I20260812 14:46:18.784384 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:18.784275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.193:39597 (local address 127.0.0.1:42860)
0812 14:46:18.784319 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:18.784356 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.193:39597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 14:46:18.784421 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21960
W20260812 14:46:18.784516 21548 catalog_manager.cc:4584] TS 1276de671b874ec6b67524983295f870 (127.15.227.193:39597): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 64bc434f9c3f4a96945cc19d3e12b695: Network error: Client connection negotiation failed: client connection to 127.15.227.193:39597: connect: Connection refused (error 111)
I20260812 14:46:18.798884 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 22228
I20260812 14:46:18.810173 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 21524
2026-08-12T14:46:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (661 ms)
[----------] 6 tests from EncryptionPolicyTest (8513 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:18 dist-test-slave-q9h9 krb5kdc[22566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:18 dist-test-slave-q9h9 krb5kdc[22566](info): set up 2 sockets
Aug 12 14:46:18 dist-test-slave-q9h9 krb5kdc[22566](info): commencing operation
krb5kdc: starting...
W20260812 14:46:19.999725 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:20 dist-test-slave-q9h9 krb5kdc[22566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:20.110322 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:20.188942 22584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:20.189129 22584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:20.189158 22584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:20.190299 22584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:20.190337 22584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:20.190356 22584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:20.190371 22584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:20.190387 22584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:20.191811 22584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37039
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:20.192045 22584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:20.192252 22584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:20.196900 22592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:20.196944 22593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:20.196907 22595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:20.197101 22584 server_base.cc:1034] running on GCE node
I20260812 14:46:20.197324 22584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:20.197553 22584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:20.198681 22584 hybrid_clock.cc:648] HybridClock initialized: now 1786545980198666 us; error 30 us; skew 500 ppm
Aug 12 14:46:20 dist-test-slave-q9h9 krb5kdc[22566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545980, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:20.200958 22584 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:20.201330 22584 webserver.cc:460] Webserver started at http://127.0.0.1:42191/ using document root <none> and password file <none>
I20260812 14:46:20.201514 22584 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:20.201553 22584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:20.201643 22584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:20.202605 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "3c02cc11717244a8812d17dc51055542"
format_stamp: "Formatted at 2026-08-12 14:46:20 on dist-test-slave-q9h9"
server_key: "8376e51e7c1b72c3a4092ad1a2b6b19a"
server_key_iv: "0a1dca39a194b3d687fd29e1231a50ff"
server_key_version: "encryptionkey@0"
I20260812 14:46:20.202881 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "3c02cc11717244a8812d17dc51055542"
format_stamp: "Formatted at 2026-08-12 14:46:20 on dist-test-slave-q9h9"
server_key: "8376e51e7c1b72c3a4092ad1a2b6b19a"
server_key_iv: "0a1dca39a194b3d687fd29e1231a50ff"
server_key_version: "encryptionkey@0"
I20260812 14:46:20.204041 22584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:20.204699 22607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:20.204843 22584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:20.204926 22584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "3c02cc11717244a8812d17dc51055542"
format_stamp: "Formatted at 2026-08-12 14:46:20 on dist-test-slave-q9h9"
server_key: "8376e51e7c1b72c3a4092ad1a2b6b19a"
server_key_iv: "0a1dca39a194b3d687fd29e1231a50ff"
server_key_version: "encryptionkey@0"
I20260812 14:46:20.204982 22584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:20.228500 22584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:20.230448 22584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:20.230574 22584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:20.234361 22584 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37039
I20260812 14:46:20.234398 22698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37039 every 8 connection(s)
I20260812 14:46:20.234688 22584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:20.235502 22699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:20.237617 22699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542: Bootstrap starting.
I20260812 14:46:20.238255 22699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:20.238518 22699 log.cc:824] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:20.239192 22699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542: No bootstrap required, opened a new log
I20260812 14:46:20.240726 22699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c02cc11717244a8812d17dc51055542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260812 14:46:20.240854 22699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:20.240885 22699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c02cc11717244a8812d17dc51055542, State: Initialized, Role: FOLLOWER
I20260812 14:46:20.241019 22699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c02cc11717244a8812d17dc51055542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260812 14:46:20.241077 22699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:20.241109 22699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:20.241155 22699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:20.241647 22699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c02cc11717244a8812d17dc51055542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260812 14:46:20.241736 22699 leader_election.cc:302] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c02cc11717244a8812d17dc51055542; no voters:
I20260812 14:46:20.241884 22699 leader_election.cc:288] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:20.241922 22709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:20.242101 22709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [term 1 LEADER]: Becoming Leader. State: Replica: 3c02cc11717244a8812d17dc51055542, State: Running, Role: LEADER
I20260812 14:46:20.242159 22699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:20.242195 22709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c02cc11717244a8812d17dc51055542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } }
I20260812 14:46:20.242547 22712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c02cc11717244a8812d17dc51055542. Latest consensus state: current_term: 1 leader_uuid: "3c02cc11717244a8812d17dc51055542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c02cc11717244a8812d17dc51055542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } } }
I20260812 14:46:20.242630 22712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:20.242770 22710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c02cc11717244a8812d17dc51055542" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c02cc11717244a8812d17dc51055542" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37039 } } }
I20260812 14:46:20.242885 22710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:20.243024 22721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:20.243536 22721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:20.244439 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 22584
I20260812 14:46:20.244566 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:20.244764 16271 external_mini_cluster.cc:1428] Setting key a95ccf34563158e98e2300fb889c9bb0
I20260812 14:46:20.244922 22721 catalog_manager.cc:1347] Generated new cluster ID: e3b69b66be254710bc1381c150468e14
I20260812 14:46:20.244966 22721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:46:20 dist-test-slave-q9h9 krb5kdc[22566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:46:20.258136 22721 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:20.258590 22721 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:20.260600 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:20.245194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54535 (local address 127.0.0.1:37039)
0812 14:46:20.250163 (+ 4969us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:20.250171 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:20.250203 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:46:20.250305 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:20.250312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:20.250370 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:20.250499 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:20.251245 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:20.252618 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:20.253683 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:20.254318 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:20.254410 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:20.257020 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:20.257025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:20.257025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:20.257036 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:20.258988 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:20.259404 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:20.259407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:20.259408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:20.259514 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:20.259725 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:20.259725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:20.259726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:20.260002 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:20.260173 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:20.260262 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 14:46:20.260367 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4928,"thread_start_us":68,"threads_started":1}
I20260812 14:46:20.261584 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:20.268519 22721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c02cc11717244a8812d17dc51055542: Generated new TSK 0
I20260812 14:46:20.268666 22721 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:20.270083 22738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:20.263109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42359 (local address 127.0.0.1:37039)
0812 14:46:20.263142 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:20.263143 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:20.263154 (+ 11us) server_negotiation.cc:407] Connection header received
0812 14:46:20.263217 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:20.263217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:20.263225 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:20.263271 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:20.263840 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:20.264846 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:20.265858 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:20.266161 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:20.266211 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:20.267488 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:20.267490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:20.267490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:20.267497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:20.269038 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:20.269374 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:20.269375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:20.269375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:20.269433 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:20.269609 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:20.269609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:20.269609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:20.269725 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:20.269819 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:20.269899 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:46:20.269960 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:20.271646 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 22584
W20260812 14:46:20.289466 22564 connection.cc:535] client connection to 127.0.0.1:37039 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37039): Connection reset by peer (error 104)
2026-08-12T14:46:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:20 dist-test-slave-q9h9 krb5kdc[22767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:20 dist-test-slave-q9h9 krb5kdc[22767](info): set up 2 sockets
Aug 12 14:46:20 dist-test-slave-q9h9 krb5kdc[22767](info): commencing operation
krb5kdc: starting...
W20260812 14:46:21.475790 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:21 dist-test-slave-q9h9 krb5kdc[22767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:21.580693 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:21.660663 22790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:21.660849 22790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:21.660871 22790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:21.661984 22790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:21.662050 22790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:21.662081 22790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:21.662106 22790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:21.662132 22790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:21.663604 22790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:21.663829 22790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:21.664021 22790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:21.668498 22806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:21.668529 22805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:21.668644 22790 server_base.cc:1034] running on GCE node
W20260812 14:46:21.668519 22814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:21.668943 22790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:21.669188 22790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:21.670320 22790 hybrid_clock.cc:648] HybridClock initialized: now 1786545981670304 us; error 31 us; skew 500 ppm
Aug 12 14:46:21 dist-test-slave-q9h9 krb5kdc[22767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545981, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:21.678254 22790 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:21.678668 22790 webserver.cc:460] Webserver started at http://127.0.0.1:43191/ using document root <none> and password file <none>
I20260812 14:46:21.678875 22790 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:21.678923 22790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:21.679047 22790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:21.680001 22790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "9204eca2d4cf4f2bb148a279f1bc3474"
format_stamp: "Formatted at 2026-08-12 14:46:21 on dist-test-slave-q9h9"
server_key: "2494daf107357976b20da368812c1d8d"
server_key_iv: "83702af9e8888d694447cd8bb88cfc03"
server_key_version: "encryptionkey@0"
I20260812 14:46:21.680326 22790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "9204eca2d4cf4f2bb148a279f1bc3474"
format_stamp: "Formatted at 2026-08-12 14:46:21 on dist-test-slave-q9h9"
server_key: "2494daf107357976b20da368812c1d8d"
server_key_iv: "83702af9e8888d694447cd8bb88cfc03"
server_key_version: "encryptionkey@0"
I20260812 14:46:21.681514 22790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260812 14:46:21.682164 22829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:21.682298 22790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:21.682381 22790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "9204eca2d4cf4f2bb148a279f1bc3474"
format_stamp: "Formatted at 2026-08-12 14:46:21 on dist-test-slave-q9h9"
server_key: "2494daf107357976b20da368812c1d8d"
server_key_iv: "83702af9e8888d694447cd8bb88cfc03"
server_key_version: "encryptionkey@0"
I20260812 14:46:21.682446 22790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:21.697386 22790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:21.699342 22790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:21.699466 22790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:21.703225 22790 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41259
I20260812 14:46:21.703258 22922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41259 every 8 connection(s)
I20260812 14:46:21.703545 22790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:21.704310 22923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:21.704658 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 22790
I20260812 14:46:21.704764 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:21.704934 16271 external_mini_cluster.cc:1428] Setting key 0ebef0db2d1f535c98278942ab0637a7
I20260812 14:46:21.707098 22923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474: Bootstrap starting.
I20260812 14:46:21.707758 22923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:21.708050 22923 log.cc:824] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:21.708724 22923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474: No bootstrap required, opened a new log
I20260812 14:46:21.710464 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:21.705382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34471 (local address 127.0.0.1:41259)
0812 14:46:21.705561 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:21.705570 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:21.705603 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:46:21.706109 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:21.706118 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:21.706175 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:21.706326 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:21.706479 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:21.707780 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:21.708535 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:21.709103 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:21.709231 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:21.709802 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:21.709807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:21.709807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:21.710003 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:21.710066 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:21.710081 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:21.710167 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:21.710240 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 14:46:21.710266 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260812 14:46:21.711079 22923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41259 } }
I20260812 14:46:21.711249 22923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:21.711295 22923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9204eca2d4cf4f2bb148a279f1bc3474, State: Initialized, Role: FOLLOWER
I20260812 14:46:21.711400 22923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41259 } }
I20260812 14:46:21.711480 22923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:21.711472 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:21.711515 22923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:21.711562 22923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:21.712055 22923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41259 } }
I20260812 14:46:21.712142 22923 leader_election.cc:302] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9204eca2d4cf4f2bb148a279f1bc3474; no voters:
I20260812 14:46:21.712296 22923 leader_election.cc:288] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:21.712368 22936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:21.712576 22936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [term 1 LEADER]: Becoming Leader. State: Replica: 9204eca2d4cf4f2bb148a279f1bc3474, State: Running, Role: LEADER
I20260812 14:46:21.712590 22923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:21.712834 22936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41259 } }
I20260812 14:46:21.713240 22947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9204eca2d4cf4f2bb148a279f1bc3474. Latest consensus state: current_term: 1 leader_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41259 } } }
I20260812 14:46:21.713269 22943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9204eca2d4cf4f2bb148a279f1bc3474" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41259 } } }
I20260812 14:46:21.713395 22943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:21.713609 22947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:21.713636 22969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:21.714152 22969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:21.715709 22969 catalog_manager.cc:1347] Generated new cluster ID: 76bbc6a9ea51476f9ad7d6852c81efdc
I20260812 14:46:21.715761 22969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:21.716239 22934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:21.712783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39411 (local address 127.0.0.1:41259)
0812 14:46:21.712816 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:21.712817 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:21.712928 (+ 111us) server_negotiation.cc:407] Connection header received
0812 14:46:21.713049 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:21.713050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:21.713059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:21.713101 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:21.713279 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:21.713948 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:21.715001 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:21.715290 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:21.715357 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:21.715951 (+ 594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:21.715952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:21.715952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:21.715956 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:21.715968 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:21.715972 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:21.716038 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:21.716163 (+ 125us) server_negotiation.cc:299] Negotiation successful
0812 14:46:21.716174 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:21.745007 22969 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:21.745473 22969 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:21.759248 22969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9204eca2d4cf4f2bb148a279f1bc3474: Generated new TSK 0
I20260812 14:46:21.759400 22969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:21.766707 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 22790
W20260812 14:46:21.784420 22757 connection.cc:535] client connection to 127.0.0.1:41259 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41259): Connection reset by peer (error 104)
2026-08-12T14:46:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:21 dist-test-slave-q9h9 krb5kdc[22997](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:21 dist-test-slave-q9h9 krb5kdc[22997](info): set up 2 sockets
Aug 12 14:46:21 dist-test-slave-q9h9 krb5kdc[22997](info): commencing operation
krb5kdc: starting...
W20260812 14:46:22.958738 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:23 dist-test-slave-q9h9 krb5kdc[22997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:23.086390 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45841
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:23.165151 23020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:23.165324 23020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:23.165345 23020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:23.166494 23020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:23.166543 23020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:23.166563 23020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:23.166580 23020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:23.166596 23020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:23.168032 23020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45841
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:23.168337 23020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:23.168545 23020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:23.172978 23026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:23.173095 23032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:23.173158 23020 server_base.cc:1034] running on GCE node
W20260812 14:46:23.173020 23029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:23.173418 23020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:23.173676 23020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:23.174819 23020 hybrid_clock.cc:648] HybridClock initialized: now 1786545983174804 us; error 29 us; skew 500 ppm
Aug 12 14:46:23 dist-test-slave-q9h9 krb5kdc[22997](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545983, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:23.177021 23020 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:23.177426 23020 webserver.cc:460] Webserver started at http://127.0.0.1:42505/ using document root <none> and password file <none>
I20260812 14:46:23.177640 23020 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:23.177687 23020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:23.177807 23020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:23.178781 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "897acf56150d4f6da6b50352f6a88b08"
format_stamp: "Formatted at 2026-08-12 14:46:23 on dist-test-slave-q9h9"
server_key: "401892eda5a8ac1b9f8354c548d3446b"
server_key_iv: "27bada91ce061aa9d7c0ea69d7637549"
server_key_version: "encryptionkey@0"
I20260812 14:46:23.179095 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "897acf56150d4f6da6b50352f6a88b08"
format_stamp: "Formatted at 2026-08-12 14:46:23 on dist-test-slave-q9h9"
server_key: "401892eda5a8ac1b9f8354c548d3446b"
server_key_iv: "27bada91ce061aa9d7c0ea69d7637549"
server_key_version: "encryptionkey@0"
I20260812 14:46:23.180263 23020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:46:23.180900 23045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:23.181046 23020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:23.181115 23020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "897acf56150d4f6da6b50352f6a88b08"
format_stamp: "Formatted at 2026-08-12 14:46:23 on dist-test-slave-q9h9"
server_key: "401892eda5a8ac1b9f8354c548d3446b"
server_key_iv: "27bada91ce061aa9d7c0ea69d7637549"
server_key_version: "encryptionkey@0"
I20260812 14:46:23.181180 23020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:23.184258 23020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:23.185803 23020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:23.185932 23020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:23.189597 23020 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45841
I20260812 14:46:23.189634 23144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45841 every 8 connection(s)
I20260812 14:46:23.189895 23020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:23.190129 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23020
I20260812 14:46:23.190244 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:23.190428 16271 external_mini_cluster.cc:1428] Setting key 6a32b8c78f828631b5a97eef62f96e41
I20260812 14:46:23.190904 23145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:23.192538 23151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:23.190857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43711 (local address 127.0.0.1:45841)
0812 14:46:23.191075 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:23.191083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:23.191105 (+ 22us) server_negotiation.cc:407] Connection header received
0812 14:46:23.191454 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:23.191459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:23.191482 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:23.191596 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:23.192185 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:23.192188 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:23.192188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:23.192192 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:23.192230 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:23.192239 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:23.192294 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:23.192357 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 14:46:23.192393 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":83,"threads_started":1}
I20260812 14:46:23.193406 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:23.193580 23145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08: Bootstrap starting.
I20260812 14:46:23.194161 23145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:23.194464 23145 log.cc:824] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:23.195233 23145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08: No bootstrap required, opened a new log
I20260812 14:46:23.195611 23151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:23.194661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42187 (local address 127.0.0.1:45841)
0812 14:46:23.194693 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:23.194695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:23.194800 (+ 105us) server_negotiation.cc:407] Connection header received
0812 14:46:23.194842 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:23.194842 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:23.194847 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:23.194912 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:23.195428 (+ 516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:23.195429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:23.195429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:23.195433 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:23.195445 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:23.195449 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:23.195494 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:23.195542 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 14:46:23.195554 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:46:23.197110 23145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "897acf56150d4f6da6b50352f6a88b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45841 } }
I20260812 14:46:23.197291 23145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:23.197348 23145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 897acf56150d4f6da6b50352f6a88b08, State: Initialized, Role: FOLLOWER
I20260812 14:46:23.197459 23145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "897acf56150d4f6da6b50352f6a88b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45841 } }
I20260812 14:46:23.197541 23145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:23.197572 23145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:23.197618 23145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:23.198136 23145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "897acf56150d4f6da6b50352f6a88b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45841 } }
I20260812 14:46:23.198230 23145 leader_election.cc:302] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 897acf56150d4f6da6b50352f6a88b08; no voters:
I20260812 14:46:23.198374 23145 leader_election.cc:288] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:23.198418 23165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:23.198546 23165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [term 1 LEADER]: Becoming Leader. State: Replica: 897acf56150d4f6da6b50352f6a88b08, State: Running, Role: LEADER
I20260812 14:46:23.198592 23145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:23.198696 23165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "897acf56150d4f6da6b50352f6a88b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45841 } }
I20260812 14:46:23.199059 23169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 897acf56150d4f6da6b50352f6a88b08. Latest consensus state: current_term: 1 leader_uuid: "897acf56150d4f6da6b50352f6a88b08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "897acf56150d4f6da6b50352f6a88b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45841 } } }
I20260812 14:46:23.199159 23169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:23.199309 23168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "897acf56150d4f6da6b50352f6a88b08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "897acf56150d4f6da6b50352f6a88b08" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45841 } } }
I20260812 14:46:23.199369 23168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08 [sys.catalog]: This master's current role is: LEADER
W20260812 14:46:23.200205 23196 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:46:23.200258 23196 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:46:23.200315 23178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:23.200506 23178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:23.201838 23178 catalog_manager.cc:1347] Generated new cluster ID: 00685d27040a460fa2db10aab2668c6c
I20260812 14:46:23.201889 23178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:23.225469 23178 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:23.225622 23178 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:23.228789 23178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 897acf56150d4f6da6b50352f6a88b08: Generated new TSK 0
I20260812 14:46:23.228941 23178 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:23.233737 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23020
2026-08-12T14:46:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:23 dist-test-slave-q9h9 krb5kdc[23201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:23 dist-test-slave-q9h9 krb5kdc[23201](info): set up 2 sockets
Aug 12 14:46:23 dist-test-slave-q9h9 krb5kdc[23201](info): commencing operation
krb5kdc: starting...
W20260812 14:46:24.433949 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:24 dist-test-slave-q9h9 krb5kdc[23201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:24.540048 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36227
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:24.617870 23226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:24.618064 23226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:24.618098 23226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:24.619186 23226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:24.619226 23226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:24.619244 23226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:24.619261 23226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:24.619277 23226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:24.620697 23226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:24.620910 23226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:24.621093 23226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:24.625826 23246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:24.625838 23239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:24.625881 23226 server_base.cc:1034] running on GCE node
W20260812 14:46:24.625829 23238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:24.626240 23226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:24.626499 23226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:24.627631 23226 hybrid_clock.cc:648] HybridClock initialized: now 1786545984627627 us; error 31 us; skew 500 ppm
Aug 12 14:46:24 dist-test-slave-q9h9 krb5kdc[23201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:24.629913 23226 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:24.630347 23226 webserver.cc:460] Webserver started at http://127.0.0.1:45849/ using document root <none> and password file <none>
I20260812 14:46:24.630555 23226 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:24.630604 23226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:24.630736 23226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:24.631670 23226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "74774ff83b954da4be880742749b1ac8"
format_stamp: "Formatted at 2026-08-12 14:46:24 on dist-test-slave-q9h9"
server_key: "2d228d535bbc13157556849ca8ce42d9"
server_key_iv: "c23e1961be45d063c8b693ebec25f1e3"
server_key_version: "encryptionkey@0"
I20260812 14:46:24.631981 23226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "74774ff83b954da4be880742749b1ac8"
format_stamp: "Formatted at 2026-08-12 14:46:24 on dist-test-slave-q9h9"
server_key: "2d228d535bbc13157556849ca8ce42d9"
server_key_iv: "c23e1961be45d063c8b693ebec25f1e3"
server_key_version: "encryptionkey@0"
I20260812 14:46:24.633236 23226 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:46:24.634032 23257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:24.634179 23226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:46:24.634269 23226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "74774ff83b954da4be880742749b1ac8"
format_stamp: "Formatted at 2026-08-12 14:46:24 on dist-test-slave-q9h9"
server_key: "2d228d535bbc13157556849ca8ce42d9"
server_key_iv: "c23e1961be45d063c8b693ebec25f1e3"
server_key_version: "encryptionkey@0"
I20260812 14:46:24.634338 23226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:24.655475 23226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:24.656989 23226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:24.657114 23226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:24.660778 23226 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36227
I20260812 14:46:24.660808 23358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36227 every 8 connection(s)
I20260812 14:46:24.661109 23226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:24.661866 23362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:24.663866 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23226
I20260812 14:46:24.663967 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:24.663975 23362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8: Bootstrap starting.
I20260812 14:46:24.664126 16271 external_mini_cluster.cc:1428] Setting key 0708a7797196393f5f7caeb682e468f3
I20260812 14:46:24.664631 23362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:24.665015 23362 log.cc:824] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:24.665711 23362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8: No bootstrap required, opened a new log
I20260812 14:46:24.667910 23362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74774ff83b954da4be880742749b1ac8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36227 } }
I20260812 14:46:24.668035 23362 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:24.668056 23362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74774ff83b954da4be880742749b1ac8, State: Initialized, Role: FOLLOWER
I20260812 14:46:24.668155 23362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74774ff83b954da4be880742749b1ac8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36227 } }
I20260812 14:46:24.668232 23362 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:24.668257 23362 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:24.668339 23362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:24.668846 23362 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74774ff83b954da4be880742749b1ac8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36227 } }
I20260812 14:46:24.668939 23362 leader_election.cc:302] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74774ff83b954da4be880742749b1ac8; no voters:
I20260812 14:46:24.669096 23362 leader_election.cc:288] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:24.669132 23368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:24.669323 23368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [term 1 LEADER]: Becoming Leader. State: Replica: 74774ff83b954da4be880742749b1ac8, State: Running, Role: LEADER
I20260812 14:46:24.669420 23362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:24.669462 23368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74774ff83b954da4be880742749b1ac8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36227 } }
I20260812 14:46:24.669876 23368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74774ff83b954da4be880742749b1ac8. Latest consensus state: current_term: 1 leader_uuid: "74774ff83b954da4be880742749b1ac8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74774ff83b954da4be880742749b1ac8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36227 } } }
I20260812 14:46:24.669936 23369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74774ff83b954da4be880742749b1ac8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74774ff83b954da4be880742749b1ac8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36227 } } }
I20260812 14:46:24.670001 23368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:24.670042 23369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:24.670308 23388 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 12 14:46:24 dist-test-slave-q9h9 krb5kdc[23201](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:46:24.671041 23388 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:24.672552 23388 catalog_manager.cc:1347] Generated new cluster ID: 77c47c992d7349f9917c867f2ccaed29
I20260812 14:46:24.672602 23388 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:24.674633 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:24.664579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34353 (local address 127.0.0.1:36227)
0812 14:46:24.664760 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:24.664769 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:24.664803 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:46:24.665332 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:24.665338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:24.665385 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:24.665528 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:24.666144 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:24.667484 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:24.668297 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:24.668731 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:24.668833 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:24.671482 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:24.671486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:24.671487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:24.671497 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:24.673232 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:24.673545 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:24.673548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:24.673548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:24.673633 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:24.673810 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:24.673811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:24.673812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:24.674056 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:24.674205 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:24.674311 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 14:46:24.674436 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260812 14:46:24.675426 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:24.681911 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:24.676643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43887 (local address 127.0.0.1:36227)
0812 14:46:24.676678 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:24.676680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:24.676794 (+ 114us) server_negotiation.cc:407] Connection header received
0812 14:46:24.676848 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:24.676848 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:24.676857 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:24.676907 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:24.677359 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:24.677809 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:24.678506 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:24.678740 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:24.678793 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:24.679812 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:24.679813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:24.679814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:24.679820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:24.680986 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:24.681283 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:24.681283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:24.681284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:24.681335 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:24.681494 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:24.681495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:24.681496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:24.681590 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:24.681669 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:24.681773 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 14:46:24.681837 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:24.685633 23388 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:24.686163 23388 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:24.700074 23388 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74774ff83b954da4be880742749b1ac8: Generated new TSK 0
I20260812 14:46:24.700260 23388 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:24.707089 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23226
2026-08-12T14:46:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1474 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:24 dist-test-slave-q9h9 krb5kdc[23424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:24 dist-test-slave-q9h9 krb5kdc[23424](info): set up 2 sockets
Aug 12 14:46:24 dist-test-slave-q9h9 krb5kdc[23424](info): commencing operation
krb5kdc: starting...
W20260812 14:46:25.908789 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:25 dist-test-slave-q9h9 krb5kdc[23424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:26.017220 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:26.096768 23448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:26.096961 23448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:26.096990 23448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:26.098122 23448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:26.098160 23448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:26.098179 23448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:26.098196 23448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:26.098210 23448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:26.099655 23448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:26.099880 23448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:26.100077 23448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:26.104471 23456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:26.104576 23448 server_base.cc:1034] running on GCE node
W20260812 14:46:26.104590 23458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:26.104472 23463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:26.104936 23448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:26.105221 23448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:26.106355 23448 hybrid_clock.cc:648] HybridClock initialized: now 1786545986106342 us; error 30 us; skew 500 ppm
Aug 12 14:46:26 dist-test-slave-q9h9 krb5kdc[23424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545986, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:26.108651 23448 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:26.109074 23448 webserver.cc:460] Webserver started at http://127.0.0.1:42949/ using document root <none> and password file <none>
I20260812 14:46:26.109280 23448 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:26.109328 23448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:26.109449 23448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:26.110466 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "a5683ba6a58947418a901527316f3833"
format_stamp: "Formatted at 2026-08-12 14:46:26 on dist-test-slave-q9h9"
server_key: "43eb953f66a203a20178ab881ccafdf8"
server_key_iv: "c28cf7b6127a597ae8b4a86ab3e6310d"
server_key_version: "encryptionkey@0"
I20260812 14:46:26.110785 23448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "a5683ba6a58947418a901527316f3833"
format_stamp: "Formatted at 2026-08-12 14:46:26 on dist-test-slave-q9h9"
server_key: "43eb953f66a203a20178ab881ccafdf8"
server_key_iv: "c28cf7b6127a597ae8b4a86ab3e6310d"
server_key_version: "encryptionkey@0"
I20260812 14:46:26.112236 23448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:26.112886 23477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:26.113046 23448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:26.113139 23448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "a5683ba6a58947418a901527316f3833"
format_stamp: "Formatted at 2026-08-12 14:46:26 on dist-test-slave-q9h9"
server_key: "43eb953f66a203a20178ab881ccafdf8"
server_key_iv: "c28cf7b6127a597ae8b4a86ab3e6310d"
server_key_version: "encryptionkey@0"
I20260812 14:46:26.113199 23448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:26.148797 23448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:26.150544 23448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:26.150671 23448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:26.154328 23448 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44965
I20260812 14:46:26.154369 23570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44965 every 8 connection(s)
I20260812 14:46:26.154662 23448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:26.155447 23571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:26.157557 23571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833: Bootstrap starting.
I20260812 14:46:26.158121 23571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:26.158389 23571 log.cc:824] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:26.158996 23571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833: No bootstrap required, opened a new log
I20260812 14:46:26.160506 23571 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683ba6a58947418a901527316f3833" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44965 } }
I20260812 14:46:26.160642 23571 raft_consensus.cc:374] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:26.160677 23571 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5683ba6a58947418a901527316f3833, State: Initialized, Role: FOLLOWER
I20260812 14:46:26.160804 23571 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683ba6a58947418a901527316f3833" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44965 } }
I20260812 14:46:26.160861 23571 raft_consensus.cc:388] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:26.160892 23571 raft_consensus.cc:482] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:26.160938 23571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:26.161347 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23448
I20260812 14:46:26.161478 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:26.161528 23571 raft_consensus.cc:504] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683ba6a58947418a901527316f3833" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44965 } }
I20260812 14:46:26.161640 23571 leader_election.cc:302] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a5683ba6a58947418a901527316f3833; no voters:
I20260812 14:46:26.161661 16271 external_mini_cluster.cc:1428] Setting key 69c1bf154c8829882b5281a236e0d7d2
I20260812 14:46:26.161794 23571 leader_election.cc:288] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:26.161830 23578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:26.162009 23578 raft_consensus.cc:686] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [term 1 LEADER]: Becoming Leader. State: Replica: a5683ba6a58947418a901527316f3833, State: Running, Role: LEADER
I20260812 14:46:26.162060 23571 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:26.162134 23578 consensus_queue.cc:237] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683ba6a58947418a901527316f3833" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44965 } }
I20260812 14:46:26.162490 23583 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a5683ba6a58947418a901527316f3833. Latest consensus state: current_term: 1 leader_uuid: "a5683ba6a58947418a901527316f3833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683ba6a58947418a901527316f3833" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44965 } } }
I20260812 14:46:26.162497 23580 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a5683ba6a58947418a901527316f3833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a5683ba6a58947418a901527316f3833" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44965 } } }
I20260812 14:46:26.162581 23583 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:26.162590 23580 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:26.162771 23592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:26.163313 23592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:26.164785 23592 catalog_manager.cc:1347] Generated new cluster ID: 359bdd61d43841a2bc85e106f933e948
I20260812 14:46:26.164830 23592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:26.167438 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:26.162186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37825 (local address 127.0.0.1:44965)
0812 14:46:26.162433 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:26.162443 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:26.162708 (+ 265us) server_negotiation.cc:407] Connection header received
0812 14:46:26.162881 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:26.162890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:26.162951 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:26.163098 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:26.163442 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:26.164691 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:26.165503 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:26.166086 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:26.166176 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:26.166786 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:26.166790 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:26.166790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:26.166986 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:26.167019 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:26.167032 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:26.167091 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:26.167190 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 14:46:26.167253 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":68,"threads_started":1}
I20260812 14:46:26.168463 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:26.173231 23588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:26.169985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46039 (local address 127.0.0.1:44965)
0812 14:46:26.170042 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:26.170043 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:26.170051 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:26.170088 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:26.170089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:26.170098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:26.170144 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:26.170263 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:26.170949 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:26.172038 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:26.172349 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:26.172392 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:26.173017 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:26.173019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:26.173019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:26.173024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:26.173037 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:26.173041 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:26.173083 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:26.173146 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 14:46:26.173167 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:46:26.186393 23592 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:26.186846 23592 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:26.198462 23592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a5683ba6a58947418a901527316f3833: Generated new TSK 0
I20260812 14:46:26.198611 23592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:26.202458 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23448
2026-08-12T14:46:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:26 dist-test-slave-q9h9 krb5kdc[23637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:26 dist-test-slave-q9h9 krb5kdc[23637](info): set up 2 sockets
Aug 12 14:46:26 dist-test-slave-q9h9 krb5kdc[23637](info): commencing operation
krb5kdc: starting...
W20260812 14:46:27.402441 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:27 dist-test-slave-q9h9 krb5kdc[23637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:27.507632 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:27.584962 23665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:27.585143 23665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:27.585165 23665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:27.586222 23665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:27.586253 23665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:27.586266 23665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:27.586277 23665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:27.586290 23665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:27.587656 23665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:27.587826 23665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:27.587988 23665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:27.592338 23680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:27.592350 23677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:27.592379 23665 server_base.cc:1034] running on GCE node
W20260812 14:46:27.592391 23675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:27.592758 23665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:27.593017 23665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:27.594151 23665 hybrid_clock.cc:648] HybridClock initialized: now 1786545987594137 us; error 31 us; skew 500 ppm
Aug 12 14:46:27 dist-test-slave-q9h9 krb5kdc[23637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:27.601023 23665 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:27.601398 23665 webserver.cc:460] Webserver started at http://127.0.0.1:38975/ using document root <none> and password file <none>
I20260812 14:46:27.601600 23665 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:27.601647 23665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:27.601775 23665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:27.602759 23665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "9bf0790076344b2f8b2c21cb58b55ee4"
format_stamp: "Formatted at 2026-08-12 14:46:27 on dist-test-slave-q9h9"
server_key: "e18f598d99049a2b9ad526bcbf2fc068"
server_key_iv: "244d033d1784b8aad0be3a03d976e8cd"
server_key_version: "encryptionkey@0"
I20260812 14:46:27.603066 23665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "9bf0790076344b2f8b2c21cb58b55ee4"
format_stamp: "Formatted at 2026-08-12 14:46:27 on dist-test-slave-q9h9"
server_key: "e18f598d99049a2b9ad526bcbf2fc068"
server_key_iv: "244d033d1784b8aad0be3a03d976e8cd"
server_key_version: "encryptionkey@0"
I20260812 14:46:27.604233 23665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260812 14:46:27.604838 23690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:27.604981 23665 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:27.605060 23665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "9bf0790076344b2f8b2c21cb58b55ee4"
format_stamp: "Formatted at 2026-08-12 14:46:27 on dist-test-slave-q9h9"
server_key: "e18f598d99049a2b9ad526bcbf2fc068"
server_key_iv: "244d033d1784b8aad0be3a03d976e8cd"
server_key_version: "encryptionkey@0"
I20260812 14:46:27.605124 23665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:27.608482 23665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:27.610073 23665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:27.610205 23665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:27.613790 23665 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35655
I20260812 14:46:27.613826 23797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35655 every 8 connection(s)
I20260812 14:46:27.614140 23665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:27.615022 23798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:27.616902 23798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4: Bootstrap starting.
I20260812 14:46:27.617386 23798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:27.617610 23798 log.cc:824] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:27.618208 23798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4: No bootstrap required, opened a new log
I20260812 14:46:27.619670 23798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260812 14:46:27.619789 23798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:27.619817 23798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bf0790076344b2f8b2c21cb58b55ee4, State: Initialized, Role: FOLLOWER
I20260812 14:46:27.619915 23798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260812 14:46:27.619989 23798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:27.620016 23798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:27.620047 23798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:27.620479 23798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260812 14:46:27.620558 23798 leader_election.cc:302] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bf0790076344b2f8b2c21cb58b55ee4; no voters:
I20260812 14:46:27.620678 23798 leader_election.cc:288] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:27.620736 23806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:27.620880 23798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:27.620886 23806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [term 1 LEADER]: Becoming Leader. State: Replica: 9bf0790076344b2f8b2c21cb58b55ee4, State: Running, Role: LEADER
I20260812 14:46:27.621032 23806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } }
I20260812 14:46:27.621189 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23665
I20260812 14:46:27.621312 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:27.621366 23808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bf0790076344b2f8b2c21cb58b55ee4. Latest consensus state: current_term: 1 leader_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } } }
I20260812 14:46:27.621438 23808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:27.621508 16271 external_mini_cluster.cc:1428] Setting key cba573a7b32eb001b0ff0c969505ea42
I20260812 14:46:27.621565 23807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0790076344b2f8b2c21cb58b55ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35655 } } }
I20260812 14:46:27.621614 23807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:27.621869 23815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:27.622381 23815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:27.623977 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:27.621895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35891 (local address 127.0.0.1:35655)
0812 14:46:27.622125 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:27.622135 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:27.622381 (+ 246us) server_negotiation.cc:407] Connection header received
0812 14:46:27.622587 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:27.622596 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:27.622630 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:27.622741 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:27.623578 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:27.623582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:27.623582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:27.623586 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:27.623626 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:27.623639 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:27.623710 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:27.623781 (+ 71us) server_negotiation.cc:299] Negotiation successful
0812 14:46:27.623818 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":68,"threads_started":1}
I20260812 14:46:27.624097 23815 catalog_manager.cc:1347] Generated new cluster ID: 6c9aa99062af4509bd939ca90cca3a19
I20260812 14:46:27.624135 23815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:27.624722 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:27.627235 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:27.625971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56451 (local address 127.0.0.1:35655)
0812 14:46:27.626003 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:27.626005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:27.626299 (+ 294us) server_negotiation.cc:407] Connection header received
0812 14:46:27.626395 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:27.626396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:27.626402 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:27.626429 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:27.626986 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:27.626987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:27.626988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:27.626992 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:27.627005 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:27.627009 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:27.627038 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:27.627151 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 14:46:27.627163 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:46:27.634816 23815 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:27.634959 23815 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:27.639012 23815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bf0790076344b2f8b2c21cb58b55ee4: Generated new TSK 0
I20260812 14:46:27.639154 23815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:27.641621 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23665
2026-08-12T14:46:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:27 dist-test-slave-q9h9 krb5kdc[23861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:27 dist-test-slave-q9h9 krb5kdc[23861](info): set up 2 sockets
Aug 12 14:46:27 dist-test-slave-q9h9 krb5kdc[23861](info): commencing operation
krb5kdc: starting...
W20260812 14:46:28.843041 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:28 dist-test-slave-q9h9 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:28.949894 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:29.029224 23894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:29.029410 23894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:29.029441 23894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:29.030560 23894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:29.030599 23894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:29.030618 23894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:29.030634 23894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:29.030650 23894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:29.032068 23894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:29.032303 23894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:29.032496 23894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:29.037256 23904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:29.037268 23906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:29.037276 23903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:29.037361 23894 server_base.cc:1034] running on GCE node
I20260812 14:46:29.037635 23894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:29.037899 23894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:29.039037 23894 hybrid_clock.cc:648] HybridClock initialized: now 1786545989039022 us; error 28 us; skew 500 ppm
Aug 12 14:46:29 dist-test-slave-q9h9 krb5kdc[23861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:29.041311 23894 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:29.041709 23894 webserver.cc:460] Webserver started at http://127.0.0.1:37425/ using document root <none> and password file <none>
I20260812 14:46:29.041886 23894 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:29.041924 23894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:29.042040 23894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:29.043009 23894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "b88b17cc400040d98a4348a9d53f45d6"
format_stamp: "Formatted at 2026-08-12 14:46:29 on dist-test-slave-q9h9"
server_key: "8752ab0db4132a781078cb9420e4717f"
server_key_iv: "273d20de8ee5aaefaeea6a857509c7c2"
server_key_version: "encryptionkey@0"
I20260812 14:46:29.043290 23894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "b88b17cc400040d98a4348a9d53f45d6"
format_stamp: "Formatted at 2026-08-12 14:46:29 on dist-test-slave-q9h9"
server_key: "8752ab0db4132a781078cb9420e4717f"
server_key_iv: "273d20de8ee5aaefaeea6a857509c7c2"
server_key_version: "encryptionkey@0"
I20260812 14:46:29.044474 23894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:29.045143 23920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:29.045300 23894 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:29.045384 23894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "b88b17cc400040d98a4348a9d53f45d6"
format_stamp: "Formatted at 2026-08-12 14:46:29 on dist-test-slave-q9h9"
server_key: "8752ab0db4132a781078cb9420e4717f"
server_key_iv: "273d20de8ee5aaefaeea6a857509c7c2"
server_key_version: "encryptionkey@0"
I20260812 14:46:29.045440 23894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:29.056610 23894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:29.058166 23894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:29.058293 23894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:29.061975 23894 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34021
I20260812 14:46:29.062038 24032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34021 every 8 connection(s)
I20260812 14:46:29.062315 23894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:29.063073 24033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:29.063653 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 23894
I20260812 14:46:29.063766 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:29.063923 16271 external_mini_cluster.cc:1428] Setting key ad7881279e3900523a52e1be0ace5b55
I20260812 14:46:29.065903 24033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6: Bootstrap starting.
I20260812 14:46:29.066623 24033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:29.066926 24033 log.cc:824] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:29.067557 24033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6: No bootstrap required, opened a new log
I20260812 14:46:29.069594 24033 raft_consensus.cc:348] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b17cc400040d98a4348a9d53f45d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34021 } }
I20260812 14:46:29.069725 24033 raft_consensus.cc:374] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:29.069746 24033 raft_consensus.cc:729] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88b17cc400040d98a4348a9d53f45d6, State: Initialized, Role: FOLLOWER
I20260812 14:46:29.069844 24033 consensus_queue.cc:260] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b17cc400040d98a4348a9d53f45d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34021 } }
I20260812 14:46:29.069909 24033 raft_consensus.cc:388] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:29.069931 24033 raft_consensus.cc:482] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:29.069976 24033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 0 FOLLOWER]: Advancing to term 1
Aug 12 14:46:29 dist-test-slave-q9h9 krb5kdc[23861](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:46:29.070524 24033 raft_consensus.cc:504] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b17cc400040d98a4348a9d53f45d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34021 } }
I20260812 14:46:29.070614 24033 leader_election.cc:302] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b88b17cc400040d98a4348a9d53f45d6; no voters:
I20260812 14:46:29.070771 24033 leader_election.cc:288] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:29.070809 24045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:29.071017 24045 raft_consensus.cc:686] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [term 1 LEADER]: Becoming Leader. State: Replica: b88b17cc400040d98a4348a9d53f45d6, State: Running, Role: LEADER
I20260812 14:46:29.071026 24033 sys_catalog.cc:564] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:29.071116 24045 consensus_queue.cc:237] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b17cc400040d98a4348a9d53f45d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34021 } }
I20260812 14:46:29.071484 24049 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b88b17cc400040d98a4348a9d53f45d6. Latest consensus state: current_term: 1 leader_uuid: "b88b17cc400040d98a4348a9d53f45d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b17cc400040d98a4348a9d53f45d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34021 } } }
I20260812 14:46:29.071856 24049 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:29.072188 24046 sys_catalog.cc:455] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b88b17cc400040d98a4348a9d53f45d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b88b17cc400040d98a4348a9d53f45d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34021 } } }
I20260812 14:46:29.072273 24046 sys_catalog.cc:458] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:29.073182 24058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:29.073642 24058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:29.074242 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:29.064452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:49963 (local address 127.0.0.1:34021)
0812 14:46:29.064633 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:29.064642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:29.064675 (+ 33us) server_negotiation.cc:407] Connection header received
0812 14:46:29.065161 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:29.065170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:29.065246 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:29.065383 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:29.065922 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:29.067245 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:29.067983 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:29.068576 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:29.068694 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:29.070779 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:29.070783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:29.070784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:29.070794 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:29.072551 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:29.072928 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:29.072932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:29.072932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:29.073022 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:29.073251 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:29.073252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:29.073252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:29.073498 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:29.073658 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:29.073974 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 14:46:29.074102 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260812 14:46:29.075304 24058 catalog_manager.cc:1347] Generated new cluster ID: 8d144599a4674c2c91e68b4f01028f4f
I20260812 14:46:29.075353 24058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:29.075469 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:29.084120 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:29.076932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:52881 (local address 127.0.0.1:34021)
0812 14:46:29.076964 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:29.076965 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:29.077184 (+ 219us) server_negotiation.cc:407] Connection header received
0812 14:46:29.077271 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:29.077272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:29.077281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:29.077307 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:29.077966 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:29.078686 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:29.079714 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:29.080049 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:29.080090 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:29.081438 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:29.081440 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:29.081440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:29.081446 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:29.082832 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:29.083239 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:29.083240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:29.083240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:29.083295 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:29.083535 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:29.083536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:29.083537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:29.083653 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:29.083719 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:29.083977 (+ 258us) server_negotiation.cc:299] Negotiation successful
0812 14:46:29.084042 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:46:29.096501 24058 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:29.096961 24058 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:29.100630 24058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b88b17cc400040d98a4348a9d53f45d6: Generated new TSK 0
I20260812 14:46:29.100778 24058 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:29.108182 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 23894
W20260812 14:46:29.138424 23859 connection.cc:535] client connection to 127.0.0.1:34021 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34021): Connection reset by peer (error 104)
2026-08-12T14:46:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1478 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:29 dist-test-slave-q9h9 krb5kdc[24095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:29 dist-test-slave-q9h9 krb5kdc[24095](info): set up 2 sockets
Aug 12 14:46:29 dist-test-slave-q9h9 krb5kdc[24095](info): commencing operation
krb5kdc: starting...
W20260812 14:46:30.312783 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.161s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:30 dist-test-slave-q9h9 krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:30.421551 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:30.500052 24117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:30.500247 24117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:30.500280 24117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:30.501370 24117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:30.501410 24117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:30.501428 24117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:30.501444 24117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:30.501461 24117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:30.502903 24117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:30.503139 24117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:30.503340 24117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:30.507678 24129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:30.507918 24117 server_base.cc:1034] running on GCE node
W20260812 14:46:30.507676 24122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:30.507700 24127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:30.508275 24117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:30.508534 24117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:30.509677 24117 hybrid_clock.cc:648] HybridClock initialized: now 1786545990509664 us; error 27 us; skew 500 ppm
Aug 12 14:46:30 dist-test-slave-q9h9 krb5kdc[24095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:30.511940 24117 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:30.512308 24117 webserver.cc:460] Webserver started at http://127.0.0.1:42207/ using document root <none> and password file <none>
I20260812 14:46:30.512504 24117 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:30.512550 24117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:30.512671 24117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:30.513583 24117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "943e12eb1d1f463cb1a831954a8b4231"
format_stamp: "Formatted at 2026-08-12 14:46:30 on dist-test-slave-q9h9"
server_key: "57ed457e73185e5aa7698bd78e654f13"
server_key_iv: "4a9015046d06ed7e31ab53ddf23f0c42"
server_key_version: "encryptionkey@0"
I20260812 14:46:30.513897 24117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "943e12eb1d1f463cb1a831954a8b4231"
format_stamp: "Formatted at 2026-08-12 14:46:30 on dist-test-slave-q9h9"
server_key: "57ed457e73185e5aa7698bd78e654f13"
server_key_iv: "4a9015046d06ed7e31ab53ddf23f0c42"
server_key_version: "encryptionkey@0"
I20260812 14:46:30.515082 24117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260812 14:46:30.515710 24142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:30.515856 24117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:30.515934 24117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "943e12eb1d1f463cb1a831954a8b4231"
format_stamp: "Formatted at 2026-08-12 14:46:30 on dist-test-slave-q9h9"
server_key: "57ed457e73185e5aa7698bd78e654f13"
server_key_iv: "4a9015046d06ed7e31ab53ddf23f0c42"
server_key_version: "encryptionkey@0"
I20260812 14:46:30.515992 24117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:30.528055 24117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:30.529533 24117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:30.529651 24117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:30.533290 24117 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39699
I20260812 14:46:30.533322 24245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39699 every 8 connection(s)
I20260812 14:46:30.533595 24117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:30.534364 24249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:30.535557 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24117
I20260812 14:46:30.535668 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:30.535830 16271 external_mini_cluster.cc:1428] Setting key 7dc76f54593274708d43a1fda44f6539
I20260812 14:46:30.536655 24249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231: Bootstrap starting.
I20260812 14:46:30.537343 24249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:30.537664 24249 log.cc:824] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:30.538393 24249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231: No bootstrap required, opened a new log
I20260812 14:46:30.540442 24249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943e12eb1d1f463cb1a831954a8b4231" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260812 14:46:30.540601 24249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:30.540650 24249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 943e12eb1d1f463cb1a831954a8b4231, State: Initialized, Role: FOLLOWER
I20260812 14:46:30.540781 24249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943e12eb1d1f463cb1a831954a8b4231" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260812 14:46:30.540866 24249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:30.540905 24249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:30.540952 24249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:30.541301 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:30.536277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:43455 (local address 127.0.0.1:39699)
0812 14:46:30.536445 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:30.536454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:30.536484 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:46:30.536970 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:30.536979 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:30.537043 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:30.537186 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:30.537361 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:30.538642 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:30.539379 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:30.539996 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:30.540120 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:30.540644 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:30.540648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:30.540649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:30.540831 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:30.540868 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:30.540884 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:30.540966 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:30.541078 (+ 112us) server_negotiation.cc:299] Negotiation successful
0812 14:46:30.541105 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260812 14:46:30.541591 24249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943e12eb1d1f463cb1a831954a8b4231" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260812 14:46:30.541695 24249 leader_election.cc:302] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 943e12eb1d1f463cb1a831954a8b4231; no voters:
I20260812 14:46:30.541850 24249 leader_election.cc:288] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:30.541872 24255 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:30.541985 24255 raft_consensus.cc:686] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [term 1 LEADER]: Becoming Leader. State: Replica: 943e12eb1d1f463cb1a831954a8b4231, State: Running, Role: LEADER
I20260812 14:46:30.542109 24249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:30.542168 24255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943e12eb1d1f463cb1a831954a8b4231" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } }
I20260812 14:46:30.542222 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:30.542536 24255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 943e12eb1d1f463cb1a831954a8b4231. Latest consensus state: current_term: 1 leader_uuid: "943e12eb1d1f463cb1a831954a8b4231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943e12eb1d1f463cb1a831954a8b4231" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } } }
I20260812 14:46:30.542758 24255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:30.542692 24256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "943e12eb1d1f463cb1a831954a8b4231" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "943e12eb1d1f463cb1a831954a8b4231" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39699 } } }
I20260812 14:46:30.543151 24256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:30.543359 24268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:30.544106 24268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:30.545483 24268 catalog_manager.cc:1347] Generated new cluster ID: 3423faed1851451d884e5a1448b31aaf
I20260812 14:46:30.545537 24268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:30.549288 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:30.543606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:45267 (local address 127.0.0.1:39699)
0812 14:46:30.543638 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:30.543640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:30.543746 (+ 106us) server_negotiation.cc:407] Connection header received
0812 14:46:30.543808 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:30.543809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:30.543816 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:30.543859 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:30.544002 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:30.544649 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:30.545484 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:30.545755 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:30.545809 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:30.549061 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:30.549062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:30.549063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:30.549067 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:30.549079 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:30.549083 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:30.549133 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:30.549209 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:46:30.549218 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:30.564647 24268 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:30.565083 24268 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:30.573807 24268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 943e12eb1d1f463cb1a831954a8b4231: Generated new TSK 0
I20260812 14:46:30.573943 24268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:30.579222 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24117
W20260812 14:46:30.592969 24091 connection.cc:535] client connection to 127.0.0.1:39699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39699): Connection reset by peer (error 104)
2026-08-12T14:46:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:30 dist-test-slave-q9h9 krb5kdc[24299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:30 dist-test-slave-q9h9 krb5kdc[24299](info): set up 2 sockets
Aug 12 14:46:30 dist-test-slave-q9h9 krb5kdc[24299](info): commencing operation
krb5kdc: starting...
W20260812 14:46:31.778771 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:31 dist-test-slave-q9h9 krb5kdc[24299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:31.884373 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:31.962522 24323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:31.962705 24323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:31.962735 24323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:31.963831 24323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:31.963871 24323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:31.963889 24323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:31.963904 24323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:31.963922 24323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:31.965361 24323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:31.965595 24323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:31.965785 24323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:31.970252 24331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:31.970265 24334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:31.970449 24332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:31.970450 24323 server_base.cc:1034] running on GCE node
I20260812 14:46:31.970688 24323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:31.970944 24323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:31.972084 24323 hybrid_clock.cc:648] HybridClock initialized: now 1786545991972072 us; error 31 us; skew 500 ppm
Aug 12 14:46:31 dist-test-slave-q9h9 krb5kdc[24299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:31.974292 24323 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:31.974709 24323 webserver.cc:460] Webserver started at http://127.0.0.1:36445/ using document root <none> and password file <none>
I20260812 14:46:31.974925 24323 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:31.974973 24323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:31.975095 24323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:31.976024 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "c0cbb86973784c93bbae78385f28996e"
format_stamp: "Formatted at 2026-08-12 14:46:31 on dist-test-slave-q9h9"
server_key: "c089aa47e01c3f75e93412e382310713"
server_key_iv: "bb83ae46080502da53a5e9a6acd70b70"
server_key_version: "encryptionkey@0"
I20260812 14:46:31.976328 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "c0cbb86973784c93bbae78385f28996e"
format_stamp: "Formatted at 2026-08-12 14:46:31 on dist-test-slave-q9h9"
server_key: "c089aa47e01c3f75e93412e382310713"
server_key_iv: "bb83ae46080502da53a5e9a6acd70b70"
server_key_version: "encryptionkey@0"
I20260812 14:46:31.977483 24323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 14:46:31.978147 24348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:31.978289 24323 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:31.978367 24323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "c0cbb86973784c93bbae78385f28996e"
format_stamp: "Formatted at 2026-08-12 14:46:31 on dist-test-slave-q9h9"
server_key: "c089aa47e01c3f75e93412e382310713"
server_key_iv: "bb83ae46080502da53a5e9a6acd70b70"
server_key_version: "encryptionkey@0"
I20260812 14:46:31.978430 24323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:31.981428 24323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:31.983376 24323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:31.983500 24323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:31.987149 24323 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39993
I20260812 14:46:31.987179 24445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39993 every 8 connection(s)
I20260812 14:46:31.987447 24323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:31.987947 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24323
I20260812 14:46:31.988031 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:31.988219 16271 external_mini_cluster.cc:1428] Setting key eaa3806dca36155fc31e38c9a81b2d39
I20260812 14:46:31.988389 24446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:31.990474 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:31.988633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:50283 (local address 127.0.0.1:39993)
0812 14:46:31.988833 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:31.988841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:31.988872 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:46:31.989400 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:31.989409 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:31.989442 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:31.989584 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:31.990070 (+ 486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:31.990073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:31.990074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:31.990078 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:31.990119 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:31.990134 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:31.990204 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:31.990262 (+ 58us) server_negotiation.cc:299] Negotiation successful
0812 14:46:31.990300 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260812 14:46:31.991358 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:31.991490 24446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e: Bootstrap starting.
I20260812 14:46:31.992028 24446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:31.992327 24446 log.cc:824] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:31.993078 24446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e: No bootstrap required, opened a new log
I20260812 14:46:31.993520 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:31.992602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:53447 (local address 127.0.0.1:39993)
0812 14:46:31.992638 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:31.992640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:31.992753 (+ 113us) server_negotiation.cc:407] Connection header received
0812 14:46:31.992813 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:31.992814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:31.992818 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:31.992862 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:31.993335 (+ 473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:31.993336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:31.993337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:31.993341 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:31.993353 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:31.993356 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:31.993402 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:31.993453 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 14:46:31.993464 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:46:31.994976 24446 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cbb86973784c93bbae78385f28996e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 14:46:31.995100 24446 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:31.995121 24446 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0cbb86973784c93bbae78385f28996e, State: Initialized, Role: FOLLOWER
I20260812 14:46:31.995201 24446 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cbb86973784c93bbae78385f28996e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 14:46:31.995270 24446 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:31.995288 24446 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:31.995317 24446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:31.995769 24446 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cbb86973784c93bbae78385f28996e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 14:46:31.996119 24446 leader_election.cc:302] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0cbb86973784c93bbae78385f28996e; no voters:
I20260812 14:46:31.996367 24446 leader_election.cc:288] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:31.996438 24466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:31.996622 24446 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:31.996663 24466 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [term 1 LEADER]: Becoming Leader. State: Replica: c0cbb86973784c93bbae78385f28996e, State: Running, Role: LEADER
I20260812 14:46:31.996845 24466 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cbb86973784c93bbae78385f28996e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } }
I20260812 14:46:31.997213 24466 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0cbb86973784c93bbae78385f28996e. Latest consensus state: current_term: 1 leader_uuid: "c0cbb86973784c93bbae78385f28996e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cbb86973784c93bbae78385f28996e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260812 14:46:31.997350 24466 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:31.997476 24468 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0cbb86973784c93bbae78385f28996e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0cbb86973784c93bbae78385f28996e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39993 } } }
I20260812 14:46:31.997612 24468 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:31.997641 24478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:31.998266 24478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:31.999925 24478 catalog_manager.cc:1347] Generated new cluster ID: 383648ca36aa4ca9a8816827f9cd55ce
I20260812 14:46:31.999977 24478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:32.004900 24478 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:32.005051 24478 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:32.010066 24478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0cbb86973784c93bbae78385f28996e: Generated new TSK 0
I20260812 14:46:32.010215 24478 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:32.012712 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24323
2026-08-12T14:46:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:32 dist-test-slave-q9h9 krb5kdc[24504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:32 dist-test-slave-q9h9 krb5kdc[24504](info): set up 2 sockets
Aug 12 14:46:32 dist-test-slave-q9h9 krb5kdc[24504](info): commencing operation
krb5kdc: starting...
W20260812 14:46:33.230746 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:33 dist-test-slave-q9h9 krb5kdc[24504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:33.338454 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34879
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:33.416894 24536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:33.417074 24536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:33.417095 24536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:33.418218 24536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:33.418257 24536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:33.418270 24536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:33.418282 24536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:33.418291 24536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:33.419732 24536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34879
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:33.419945 24536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:33.420149 24536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:33.424621 24545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:33.424727 24546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:33.424624 24548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:33.424829 24536 server_base.cc:1034] running on GCE node
I20260812 14:46:33.425040 24536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:33.425292 24536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:33.426431 24536 hybrid_clock.cc:648] HybridClock initialized: now 1786545993426416 us; error 32 us; skew 500 ppm
Aug 12 14:46:33 dist-test-slave-q9h9 krb5kdc[24504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:33.428627 24536 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:33.429028 24536 webserver.cc:460] Webserver started at http://127.0.0.1:32947/ using document root <none> and password file <none>
I20260812 14:46:33.429220 24536 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:33.429268 24536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:33.429386 24536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:33.430395 24536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "721a01b7821d439891cbac74b3b0eea2"
format_stamp: "Formatted at 2026-08-12 14:46:33 on dist-test-slave-q9h9"
server_key: "96e28ff7f48c4b333460af0d3750b623"
server_key_iv: "f99608a9b64951b551ec1b9f0eb9fdee"
server_key_version: "encryptionkey@0"
I20260812 14:46:33.430711 24536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "721a01b7821d439891cbac74b3b0eea2"
format_stamp: "Formatted at 2026-08-12 14:46:33 on dist-test-slave-q9h9"
server_key: "96e28ff7f48c4b333460af0d3750b623"
server_key_iv: "f99608a9b64951b551ec1b9f0eb9fdee"
server_key_version: "encryptionkey@0"
I20260812 14:46:33.431926 24536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:33.432611 24560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:33.432790 24536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:33.432883 24536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "721a01b7821d439891cbac74b3b0eea2"
format_stamp: "Formatted at 2026-08-12 14:46:33 on dist-test-slave-q9h9"
server_key: "96e28ff7f48c4b333460af0d3750b623"
server_key_iv: "f99608a9b64951b551ec1b9f0eb9fdee"
server_key_version: "encryptionkey@0"
I20260812 14:46:33.432947 24536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:33.464885 24536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:33.466506 24536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:33.466635 24536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:33.470397 24536 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34879
I20260812 14:46:33.470430 24657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34879 every 8 connection(s)
I20260812 14:46:33.470741 24536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:33.471529 24658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:33.472507 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24536
I20260812 14:46:33.472635 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:33.472834 16271 external_mini_cluster.cc:1428] Setting key bcc8a5dddea661191e4a85271d7a9c09
I20260812 14:46:33.474116 24658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2: Bootstrap starting.
I20260812 14:46:33.474686 24658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:33.474970 24658 log.cc:824] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:33.475596 24658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2: No bootstrap required, opened a new log
I20260812 14:46:33.477797 24658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721a01b7821d439891cbac74b3b0eea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34879 } }
I20260812 14:46:33.477967 24658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:33.478032 24658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 721a01b7821d439891cbac74b3b0eea2, State: Initialized, Role: FOLLOWER
I20260812 14:46:33.478168 24658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721a01b7821d439891cbac74b3b0eea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34879 } }
I20260812 14:46:33.478253 24658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:33.478292 24658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:33.478343 24658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:33.478950 24658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721a01b7821d439891cbac74b3b0eea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34879 } }
I20260812 14:46:33.479039 24658 leader_election.cc:302] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 721a01b7821d439891cbac74b3b0eea2; no voters:
Aug 12 14:46:33 dist-test-slave-q9h9 krb5kdc[24504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 14:46:33.479182 24658 leader_election.cc:288] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:33.479231 24668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:33.479416 24658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:33.479452 24668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [term 1 LEADER]: Becoming Leader. State: Replica: 721a01b7821d439891cbac74b3b0eea2, State: Running, Role: LEADER
I20260812 14:46:33.479637 24668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721a01b7821d439891cbac74b3b0eea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34879 } }
I20260812 14:46:33.480022 24669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "721a01b7821d439891cbac74b3b0eea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721a01b7821d439891cbac74b3b0eea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34879 } } }
I20260812 14:46:33.480098 24669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:33.480319 24672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 721a01b7821d439891cbac74b3b0eea2. Latest consensus state: current_term: 1 leader_uuid: "721a01b7821d439891cbac74b3b0eea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "721a01b7821d439891cbac74b3b0eea2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34879 } } }
I20260812 14:46:33.480384 24672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:33.480568 24686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:33.481426 24686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:33.483124 24686 catalog_manager.cc:1347] Generated new cluster ID: 8eaea4ee88b6419ba1c566c34b84dc1f
I20260812 14:46:33.483179 24686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:33.483604 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:33.473320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:54287 (local address 127.0.0.1:34879)
0812 14:46:33.473466 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:33.473473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:33.473496 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:46:33.473904 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:33.473910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:33.473956 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:33.474114 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:33.474727 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:33.475636 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:33.476671 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:33.477046 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:33.477114 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:33.479751 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:33.479755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:33.479755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:33.479765 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:33.481771 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:33.482278 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:33.482281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:33.482282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:33.482374 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:33.482601 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:33.482602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:33.482602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:33.482886 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:33.483033 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:33.483390 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 14:46:33.483474 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260812 14:46:33.486678 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:33.493471 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:33.488118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:49137 (local address 127.0.0.1:34879)
0812 14:46:33.488155 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:33.488157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:33.488301 (+ 144us) server_negotiation.cc:407] Connection header received
0812 14:46:33.488361 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:33.488362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:33.488372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:33.488421 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:33.488869 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:33.489322 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:33.489964 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:33.490237 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:33.490286 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:33.491320 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:33.491321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:33.491321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:33.491328 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:33.492485 (+ 1157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:33.492804 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:33.492805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:33.492805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:33.492856 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:33.493043 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:33.493044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:33.493044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:33.493141 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:33.493225 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:33.493341 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 14:46:33.493395 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260812 14:46:33.511540 24686 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:33.511979 24686 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:33.521054 24686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 721a01b7821d439891cbac74b3b0eea2: Generated new TSK 0
I20260812 14:46:33.521209 24686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:33.533128 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24536
W20260812 14:46:33.549515 24502 connection.cc:535] client connection to 127.0.0.1:34879 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34879): Connection reset by peer (error 104)
2026-08-12T14:46:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:33 dist-test-slave-q9h9 krb5kdc[24723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:33 dist-test-slave-q9h9 krb5kdc[24723](info): set up 2 sockets
Aug 12 14:46:33 dist-test-slave-q9h9 krb5kdc[24723](info): commencing operation
krb5kdc: starting...
W20260812 14:46:34.726567 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:34 dist-test-slave-q9h9 krb5kdc[24723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:34.833945 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:34.913651 24752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:34.913843 24752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:34.913873 24752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:34.915011 24752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:34.915050 24752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:34.915069 24752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:34.915086 24752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:34.915102 24752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:34.916532 24752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:34.916761 24752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:34.916956 24752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:34.921346 24767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:34.921372 24766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:34.921372 24771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:34.921437 24752 server_base.cc:1034] running on GCE node
I20260812 14:46:34.921731 24752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:34.921998 24752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:34.923152 24752 hybrid_clock.cc:648] HybridClock initialized: now 1786545994923143 us; error 27 us; skew 500 ppm
Aug 12 14:46:34 dist-test-slave-q9h9 krb5kdc[24723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545994, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:34.925319 24752 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:34.925684 24752 webserver.cc:460] Webserver started at http://127.0.0.1:32775/ using document root <none> and password file <none>
I20260812 14:46:34.925843 24752 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:34.925879 24752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:34.925963 24752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:34.926920 24752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5"
format_stamp: "Formatted at 2026-08-12 14:46:34 on dist-test-slave-q9h9"
server_key: "501b8e808a7b08526a1971c7805189c9"
server_key_iv: "eca8de2f25ea5054c4853f2e539e7edf"
server_key_version: "encryptionkey@0"
I20260812 14:46:34.927203 24752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5"
format_stamp: "Formatted at 2026-08-12 14:46:34 on dist-test-slave-q9h9"
server_key: "501b8e808a7b08526a1971c7805189c9"
server_key_iv: "eca8de2f25ea5054c4853f2e539e7edf"
server_key_version: "encryptionkey@0"
I20260812 14:46:34.928314 24752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:34.928934 24784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:34.929095 24752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:34.929173 24752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5"
format_stamp: "Formatted at 2026-08-12 14:46:34 on dist-test-slave-q9h9"
server_key: "501b8e808a7b08526a1971c7805189c9"
server_key_iv: "eca8de2f25ea5054c4853f2e539e7edf"
server_key_version: "encryptionkey@0"
I20260812 14:46:34.929229 24752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:34.941900 24752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:34.943418 24752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:34.943516 24752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:34.947149 24752 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34359
I20260812 14:46:34.947190 24895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34359 every 8 connection(s)
I20260812 14:46:34.947468 24752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:34.948232 24896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:34.949091 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24752
I20260812 14:46:34.949203 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:34.949363 16271 external_mini_cluster.cc:1428] Setting key 7a31a4aaa051227840335bedaa7ba3e3
I20260812 14:46:34.950773 24896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5: Bootstrap starting.
I20260812 14:46:34.951333 24896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:34.951586 24896 log.cc:824] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:34.952237 24896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5: No bootstrap required, opened a new log
I20260812 14:46:34.954397 24896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260812 14:46:34.954553 24896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:34.954561 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:34.949799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:47885 (local address 127.0.0.1:34359)
0812 14:46:34.949962 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:34.949971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:34.950001 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:46:34.950507 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:34.950515 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:34.950578 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:34.950739 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:34.950926 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:34.951849 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:34.952640 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:34.953242 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:34.953355 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:34.953901 (+ 546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:34.953905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:34.953905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:34.954140 (+ 235us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:34.954175 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:34.954190 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:34.954276 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:34.954348 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 14:46:34.954373 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260812 14:46:34.954617 24896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83ff39bc4aef4a08a6fc8ef5597a1bd5, State: Initialized, Role: FOLLOWER
I20260812 14:46:34.954895 24896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260812 14:46:34.954972 24896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:34.955003 24896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:34.955053 24896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:34.955539 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:34.955655 24896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260812 14:46:34.955758 24896 leader_election.cc:302] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83ff39bc4aef4a08a6fc8ef5597a1bd5; no voters:
I20260812 14:46:34.955904 24896 leader_election.cc:288] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:34.955945 24910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:34.956094 24910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [term 1 LEADER]: Becoming Leader. State: Replica: 83ff39bc4aef4a08a6fc8ef5597a1bd5, State: Running, Role: LEADER
I20260812 14:46:34.956126 24896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:34.956238 24910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260812 14:46:34.956553 24910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83ff39bc4aef4a08a6fc8ef5597a1bd5. Latest consensus state: current_term: 1 leader_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } } }
I20260812 14:46:34.956631 24910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:34.956743 24913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83ff39bc4aef4a08a6fc8ef5597a1bd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } } }
I20260812 14:46:34.956967 24913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:34.956970 24938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:34.957568 24938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:34.960270 24938 catalog_manager.cc:1347] Generated new cluster ID: 1db017311b484685bf479fef360d3adf
I20260812 14:46:34.960325 24938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:34.961090 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:34.956945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:45271 (local address 127.0.0.1:34359)
0812 14:46:34.956976 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:34.956977 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:34.957915 (+ 938us) server_negotiation.cc:407] Connection header received
0812 14:46:34.958001 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:34.958002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:34.958010 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:34.958054 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:34.958249 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:34.958922 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:34.959907 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:34.960117 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:34.960183 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:34.960873 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:34.960874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:34.960875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:34.960880 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:34.960895 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:34.960900 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:34.960964 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:34.961018 (+ 54us) server_negotiation.cc:299] Negotiation successful
0812 14:46:34.961028 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:46:34.971594 24938 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:34.972055 24938 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:34.975383 24938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83ff39bc4aef4a08a6fc8ef5597a1bd5: Generated new TSK 0
I20260812 14:46:34.975526 24938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:34.979585 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24752
W20260812 14:46:35.005280 24717 connection.cc:535] client connection to 127.0.0.1:34359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34359): Connection reset by peer (error 104)
2026-08-12T14:46:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:35 dist-test-slave-q9h9 krb5kdc[24968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:35 dist-test-slave-q9h9 krb5kdc[24968](info): set up 2 sockets
Aug 12 14:46:35 dist-test-slave-q9h9 krb5kdc[24968](info): commencing operation
krb5kdc: starting...
W20260812 14:46:36.196115 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:36 dist-test-slave-q9h9 krb5kdc[24968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 14:46:36.304594 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44519
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:36.384243 24994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:36.384428 24994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:36.384451 24994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:36.385509 24994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:36.385547 24994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:36.385562 24994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:36.385581 24994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:36.385598 24994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:36.387027 24994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44519
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:36.387220 24994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:36.387387 24994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:36.391876 25009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:36.392021 24994 server_base.cc:1034] running on GCE node
W20260812 14:46:36.391884 25006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:36.391907 25007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:36.392343 24994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:36.392573 24994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:36.393715 24994 hybrid_clock.cc:648] HybridClock initialized: now 1786545996393703 us; error 26 us; skew 500 ppm
Aug 12 14:46:36 dist-test-slave-q9h9 krb5kdc[24968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545996, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:36.395906 24994 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 14:46:36.396286 24994 webserver.cc:460] Webserver started at http://127.0.0.1:42851/ using document root <none> and password file <none>
I20260812 14:46:36.396466 24994 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:36.396510 24994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:36.396628 24994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:36.397539 24994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "82fc3515ac804bf29b513f075b532671"
format_stamp: "Formatted at 2026-08-12 14:46:36 on dist-test-slave-q9h9"
server_key: "40ffea736a6b5dffcd1efe4a94dcd0e0"
server_key_iv: "17f8417f51fee7316b53ee1a518a46b1"
server_key_version: "encryptionkey@0"
I20260812 14:46:36.397852 24994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "82fc3515ac804bf29b513f075b532671"
format_stamp: "Formatted at 2026-08-12 14:46:36 on dist-test-slave-q9h9"
server_key: "40ffea736a6b5dffcd1efe4a94dcd0e0"
server_key_iv: "17f8417f51fee7316b53ee1a518a46b1"
server_key_version: "encryptionkey@0"
I20260812 14:46:36.399086 24994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:36.399734 25017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:36.399878 24994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:36.399956 24994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "82fc3515ac804bf29b513f075b532671"
format_stamp: "Formatted at 2026-08-12 14:46:36 on dist-test-slave-q9h9"
server_key: "40ffea736a6b5dffcd1efe4a94dcd0e0"
server_key_iv: "17f8417f51fee7316b53ee1a518a46b1"
server_key_version: "encryptionkey@0"
I20260812 14:46:36.400019 24994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:36.403048 24994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:36.404567 24994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:36.404693 24994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:36.408274 24994 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44519
I20260812 14:46:36.408311 25112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44519 every 8 connection(s)
I20260812 14:46:36.408576 24994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:36.409469 25113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:36.411518 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671: Bootstrap starting.
I20260812 14:46:36.411993 25113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:36.412233 25113 log.cc:824] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:36.412807 25113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671: No bootstrap required, opened a new log
I20260812 14:46:36.414336 25113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fc3515ac804bf29b513f075b532671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260812 14:46:36.414459 25113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:36.414485 25113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82fc3515ac804bf29b513f075b532671, State: Initialized, Role: FOLLOWER
I20260812 14:46:36.414587 25113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fc3515ac804bf29b513f075b532671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260812 14:46:36.414655 25113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:36.414682 25113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:36.414712 25113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:36.415179 25113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fc3515ac804bf29b513f075b532671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260812 14:46:36.415268 25113 leader_election.cc:302] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82fc3515ac804bf29b513f075b532671; no voters:
I20260812 14:46:36.415390 25113 leader_election.cc:288] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:36.415439 25120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:36.415591 25113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:36.415607 25120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [term 1 LEADER]: Becoming Leader. State: Replica: 82fc3515ac804bf29b513f075b532671, State: Running, Role: LEADER
I20260812 14:46:36.415773 25120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fc3515ac804bf29b513f075b532671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } }
I20260812 14:46:36.416268 25124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82fc3515ac804bf29b513f075b532671. Latest consensus state: current_term: 1 leader_uuid: "82fc3515ac804bf29b513f075b532671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fc3515ac804bf29b513f075b532671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } } }
I20260812 14:46:36.416287 25123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82fc3515ac804bf29b513f075b532671" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82fc3515ac804bf29b513f075b532671" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44519 } } }
I20260812 14:46:36.416397 25124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:36.416404 25123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:36.416810 25135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:36.417249 25135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:36.418448 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 24994
I20260812 14:46:36.418574 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:36.418684 25135 catalog_manager.cc:1347] Generated new cluster ID: 809f0bd53dcf4bc8828c37f264cbd7ae
I20260812 14:46:36.418740 25135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:36.418772 16271 external_mini_cluster.cc:1428] Setting key 6ad5c059404177d5e734d460bef6faca
I20260812 14:46:36.420732 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:36.419168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:55291 (local address 127.0.0.1:44519)
0812 14:46:36.419323 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:36.419331 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:36.419492 (+ 161us) server_negotiation.cc:407] Connection header received
0812 14:46:36.419629 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:36.419636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:36.419663 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:36.419793 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:36.420369 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:36.420372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:36.420372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:36.420375 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:36.420405 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:36.420417 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:36.420472 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:36.420539 (+ 67us) server_negotiation.cc:299] Negotiation successful
0812 14:46:36.420575 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260812 14:46:36.421540 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:36.423907 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:36.422921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.62:36781 (local address 127.0.0.1:44519)
0812 14:46:36.422953 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:36.422954 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:36.423081 (+ 127us) server_negotiation.cc:407] Connection header received
0812 14:46:36.423147 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:36.423147 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:36.423152 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:36.423194 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:36.423711 (+ 517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:36.423712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:36.423712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:36.423716 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:36.423731 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:36.423735 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:36.423787 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:36.423840 (+ 53us) server_negotiation.cc:299] Negotiation successful
0812 14:46:36.423852 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:46:36.424937 25135 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:36.425073 25135 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:36.429136 25135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82fc3515ac804bf29b513f075b532671: Generated new TSK 0
I20260812 14:46:36.429307 25135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:36.435616 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 24994
2026-08-12T14:46:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:36 dist-test-slave-q9h9 krb5kdc[25173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:36 dist-test-slave-q9h9 krb5kdc[25173](info): set up 2 sockets
Aug 12 14:46:36 dist-test-slave-q9h9 krb5kdc[25173](info): commencing operation
krb5kdc: starting...
W20260812 14:46:37.622187 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:37 dist-test-slave-q9h9 krb5kdc[25173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:37.737551 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33579
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:45417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:37.815291 25194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:37.815475 25194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:37.815506 25194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:37.816597 25194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:37.816637 25194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:37.816656 25194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:37.816672 25194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:37.816689 25194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:37.818111 25194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:45417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:33579
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:37.818343 25194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:37.818531 25194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:37.823012 25210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:37.823041 25202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:37.823200 25194 server_base.cc:1034] running on GCE node
W20260812 14:46:37.823041 25205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:37.823490 25194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:37.823740 25194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:37.824892 25194 hybrid_clock.cc:648] HybridClock initialized: now 1786545997824879 us; error 28 us; skew 500 ppm
Aug 12 14:46:37 dist-test-slave-q9h9 krb5kdc[25173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545997, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:37.827211 25194 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:37.827596 25194 webserver.cc:460] Webserver started at http://127.15.227.254:36787/ using document root <none> and password file <none>
I20260812 14:46:37.827781 25194 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:37.827822 25194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:37.827946 25194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:37.828900 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a"
format_stamp: "Formatted at 2026-08-12 14:46:37 on dist-test-slave-q9h9"
server_key: "78816be4ba570a4e99339f6e5911438f"
server_key_iv: "7339befad9a49294180341c9b1bdb532"
server_key_version: "encryptionkey@0"
I20260812 14:46:37.829216 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a"
format_stamp: "Formatted at 2026-08-12 14:46:37 on dist-test-slave-q9h9"
server_key: "78816be4ba570a4e99339f6e5911438f"
server_key_iv: "7339befad9a49294180341c9b1bdb532"
server_key_version: "encryptionkey@0"
I20260812 14:46:37.830533 25194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:46:37.831223 25223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:37.831382 25194 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260812 14:46:37.831462 25194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a"
format_stamp: "Formatted at 2026-08-12 14:46:37 on dist-test-slave-q9h9"
server_key: "78816be4ba570a4e99339f6e5911438f"
server_key_iv: "7339befad9a49294180341c9b1bdb532"
server_key_version: "encryptionkey@0"
I20260812 14:46:37.831514 25194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:37.843000 25194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:37.844619 25194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:37.844717 25194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:37.848338 25194 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:33579
I20260812 14:46:37.848380 25317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:33579 every 8 connection(s)
I20260812 14:46:37.848654 25194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:37.849447 25319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:37.851255 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25194
I20260812 14:46:37.851361 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:37.851528 16271 external_mini_cluster.cc:1428] Setting key 52ab41ce907d2064b319b544733b69a5
I20260812 14:46:37.851698 25319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a: Bootstrap starting.
I20260812 14:46:37.852396 25319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:37.852720 25319 log.cc:824] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:37.853338 25319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a: No bootstrap required, opened a new log
I20260812 14:46:37.855157 25319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33579 } }
I20260812 14:46:37.855280 25319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:37.855311 25319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1219b2a95a1e4541bf6bf7e4ecfee23a, State: Initialized, Role: FOLLOWER
I20260812 14:46:37.855427 25319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33579 } }
I20260812 14:46:37.855499 25319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:37.855530 25319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:37.855574 25319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:37.856068 25319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33579 } }
I20260812 14:46:37.856161 25319 leader_election.cc:302] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1219b2a95a1e4541bf6bf7e4ecfee23a; no voters:
I20260812 14:46:37.856302 25319 leader_election.cc:288] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:37.856338 25327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:37.856446 25327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [term 1 LEADER]: Becoming Leader. State: Replica: 1219b2a95a1e4541bf6bf7e4ecfee23a, State: Running, Role: LEADER
I20260812 14:46:37.856540 25319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:37.856578 25327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33579 } }
I20260812 14:46:37.856909 25327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33579 } } }
I20260812 14:46:37.857013 25327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:37.857210 25327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1219b2a95a1e4541bf6bf7e4ecfee23a. Latest consensus state: current_term: 1 leader_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1219b2a95a1e4541bf6bf7e4ecfee23a" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 33579 } } }
I20260812 14:46:37.857275 25338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:37.857335 25327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:37.857853 25338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 12 14:46:37 dist-test-slave-q9h9 krb5kdc[25173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:37.860141 25338 catalog_manager.cc:1347] Generated new cluster ID: 058b521f87f0499bb8716a0789b5486d
I20260812 14:46:37.860195 25338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:37.862161 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:37.851999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58135 (local address 127.15.227.254:33579)
0812 14:46:37.852188 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:37.852197 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:37.853033 (+ 836us) server_negotiation.cc:407] Connection header received
0812 14:46:37.853469 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:37.853475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:37.853520 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:37.853637 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:37.854277 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:37.855138 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:37.855976 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:37.856430 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:37.856520 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:37.859068 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:37.859073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:37.859073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:37.859084 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:37.860753 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:37.861098 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:37.861101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:37.861102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:37.861186 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:37.861381 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:37.861382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:37.861382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:37.861606 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:37.861774 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:37.861854 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:46:37.861942 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260812 14:46:37.863058 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:37.869607 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:37.864305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37079 (local address 127.15.227.254:33579)
0812 14:46:37.864336 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:37.864338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:37.864438 (+ 100us) server_negotiation.cc:407] Connection header received
0812 14:46:37.864492 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:37.864492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:37.864501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:37.864545 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:37.864993 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:37.865459 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:37.866139 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:37.866362 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:37.866415 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:37.867409 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:37.867410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:37.867411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:37.867417 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:37.868594 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:37.868920 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:37.868921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:37.868921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:37.868972 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:37.869154 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:37.869155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:37.869156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:37.869258 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:37.869345 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:37.869477 (+ 132us) server_negotiation.cc:299] Negotiation successful
0812 14:46:37.869532 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:46:37.872512 25338 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:37.872967 25338 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:37.889248 25338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1219b2a95a1e4541bf6bf7e4ecfee23a: Generated new TSK 0
I20260812 14:46:37.889395 25338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:37.898106 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25194
W20260812 14:46:37.914289 25171 connection.cc:535] client connection to 127.15.227.254:33579 recv error: Network error: failed to read from TLS socket (remote: 127.15.227.254:33579): Connection reset by peer (error 104)
2026-08-12T14:46:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:37 dist-test-slave-q9h9 krb5kdc[25368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:37 dist-test-slave-q9h9 krb5kdc[25368](info): set up 2 sockets
Aug 12 14:46:37 dist-test-slave-q9h9 krb5kdc[25368](info): commencing operation
krb5kdc: starting...
W20260812 14:46:39.092581 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:39 dist-test-slave-q9h9 krb5kdc[25368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:39.198868 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:39215
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:41519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:39215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:39.277657 25390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:39.277835 25390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:39.277856 25390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:39.278916 25390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:39.278950 25390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:39.278962 25390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:39.278973 25390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:39.278988 25390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:39.280385 25390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:41519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:39215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:39215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:39.280562 25390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:39.280720 25390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:39.285084 25398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:39.285086 25404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:39.285100 25402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:39.285298 25390 server_base.cc:1034] running on GCE node
I20260812 14:46:39.285533 25390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:39.285785 25390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:39.286957 25390 hybrid_clock.cc:648] HybridClock initialized: now 1786545999286944 us; error 26 us; skew 500 ppm
Aug 12 14:46:39 dist-test-slave-q9h9 krb5kdc[25368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786545999, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:39.292469 25390 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:39.292881 25390 webserver.cc:460] Webserver started at http://127.15.227.254:46345/ using document root <none> and password file <none>
I20260812 14:46:39.293071 25390 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:39.293111 25390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:39.293241 25390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:39.294245 25390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "e2a1418673f54d858f67aee9ed43ca25"
format_stamp: "Formatted at 2026-08-12 14:46:39 on dist-test-slave-q9h9"
server_key: "e77fe1a14922963467f761a6118159b7"
server_key_iv: "f1f3dba9f0bbecd8396e85174afc0d97"
server_key_version: "encryptionkey@0"
I20260812 14:46:39.294550 25390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "e2a1418673f54d858f67aee9ed43ca25"
format_stamp: "Formatted at 2026-08-12 14:46:39 on dist-test-slave-q9h9"
server_key: "e77fe1a14922963467f761a6118159b7"
server_key_iv: "f1f3dba9f0bbecd8396e85174afc0d97"
server_key_version: "encryptionkey@0"
I20260812 14:46:39.295735 25390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:39.296355 25414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:39.296504 25390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:39.296586 25390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "e2a1418673f54d858f67aee9ed43ca25"
format_stamp: "Formatted at 2026-08-12 14:46:39 on dist-test-slave-q9h9"
server_key: "e77fe1a14922963467f761a6118159b7"
server_key_iv: "f1f3dba9f0bbecd8396e85174afc0d97"
server_key_version: "encryptionkey@0"
I20260812 14:46:39.296643 25390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:39.317374 25390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:39.318948 25390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:39.319070 25390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:39.322693 25390 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:39215
I20260812 14:46:39.322739 25518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:39215 every 8 connection(s)
I20260812 14:46:39.323019 25390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:39.323777 25519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:39.325853 25519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25: Bootstrap starting.
I20260812 14:46:39.326419 25519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:39.326669 25519 log.cc:824] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:39.327306 25519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25: No bootstrap required, opened a new log
I20260812 14:46:39.328819 25519 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a1418673f54d858f67aee9ed43ca25" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39215 } }
I20260812 14:46:39.328944 25519 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:39.328976 25519 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a1418673f54d858f67aee9ed43ca25, State: Initialized, Role: FOLLOWER
I20260812 14:46:39.329110 25519 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a1418673f54d858f67aee9ed43ca25" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39215 } }
I20260812 14:46:39.329186 25519 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:39.329228 25519 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:39.329277 25519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:39.330082 25519 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a1418673f54d858f67aee9ed43ca25" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39215 } }
I20260812 14:46:39.330171 25519 leader_election.cc:302] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2a1418673f54d858f67aee9ed43ca25; no voters:
I20260812 14:46:39.330327 25519 leader_election.cc:288] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:39.330353 25526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:39.330497 25526 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [term 1 LEADER]: Becoming Leader. State: Replica: e2a1418673f54d858f67aee9ed43ca25, State: Running, Role: LEADER
I20260812 14:46:39.330549 25519 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:39.330580 25526 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a1418673f54d858f67aee9ed43ca25" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39215 } }
I20260812 14:46:39.330932 25528 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2a1418673f54d858f67aee9ed43ca25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a1418673f54d858f67aee9ed43ca25" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39215 } } }
I20260812 14:46:39.330941 25530 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2a1418673f54d858f67aee9ed43ca25. Latest consensus state: current_term: 1 leader_uuid: "e2a1418673f54d858f67aee9ed43ca25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2a1418673f54d858f67aee9ed43ca25" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39215 } } }
I20260812 14:46:39.331081 25528 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:39.331089 25530 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:39.331463 25540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:39.331940 25540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:39.332780 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25390
I20260812 14:46:39.332901 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:39.333084 16271 external_mini_cluster.cc:1428] Setting key cd55cb8b6308bc1e4ddd4b8c3bab739d
I20260812 14:46:39.333421 25540 catalog_manager.cc:1347] Generated new cluster ID: 9d6463aac3f243569c540e6d2f7f7793
I20260812 14:46:39.333475 25540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:39.346171 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:39.333623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54795 (local address 127.15.227.254:39215)
0812 14:46:39.342383 (+ 8760us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:39.342390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:39.342412 (+ 22us) server_negotiation.cc:407] Connection header received
0812 14:46:39.342493 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:39.342498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:39.342535 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:39.342640 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:39.342861 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:39.343711 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:39.344452 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:39.344848 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:39.344941 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:39.345526 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:39.345530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:39.345530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:39.345689 (+ 159us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:39.345746 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:39.345762 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:39.345844 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:39.345921 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 14:46:39.345948 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8701,"thread_start_us":62,"threads_started":1}
I20260812 14:46:39.346972 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:39.350785 25561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:39.348267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33513 (local address 127.15.227.254:39215)
0812 14:46:39.348325 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:39.348327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:39.348336 (+ 9us) server_negotiation.cc:407] Connection header received
0812 14:46:39.348366 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:39.348367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:39.348375 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:39.348438 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:39.348545 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:39.349013 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:39.349681 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:39.349895 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:39.349947 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:39.350558 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:39.350559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:39.350559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:39.350563 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:39.350575 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:39.350579 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:39.350619 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:39.350694 (+ 75us) server_negotiation.cc:299] Negotiation successful
0812 14:46:39.350707 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260812 14:46:39.352743 25540 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:39.353369 25540 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:39.358273 25540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2a1418673f54d858f67aee9ed43ca25: Generated new TSK 0
I20260812 14:46:39.358422 25540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:39.366906 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25390
W20260812 14:46:39.386494 25365 connection.cc:535] client connection to 127.15.227.254:39215 recv error: Network error: failed to read from TLS socket (remote: 127.15.227.254:39215): Connection reset by peer (error 104)
2026-08-12T14:46:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:39 dist-test-slave-q9h9 krb5kdc[25573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:39 dist-test-slave-q9h9 krb5kdc[25573](info): set up 2 sockets
Aug 12 14:46:39 dist-test-slave-q9h9 krb5kdc[25573](info): commencing operation
krb5kdc: starting...
W20260812 14:46:40.566632 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:40 dist-test-slave-q9h9 krb5kdc[25573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:40.670578 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37203
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:37203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:40.749034 25594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:40.749212 25594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:40.749246 25594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:40.750396 25594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:40.750437 25594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:40.750451 25594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:40.750464 25594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:40.750489 25594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:40.751932 25594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:37203
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:37203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:40.752147 25594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:40.752343 25594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:40.756675 25603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:40.756744 25605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:40.756773 25610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:40.756845 25594 server_base.cc:1034] running on GCE node
I20260812 14:46:40.757063 25594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:40.757328 25594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:40.758466 25594 hybrid_clock.cc:648] HybridClock initialized: now 1786546000758449 us; error 29 us; skew 500 ppm
Aug 12 14:46:40 dist-test-slave-q9h9 krb5kdc[25573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546000, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:40.760624 25594 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:40.761020 25594 webserver.cc:460] Webserver started at http://127.15.227.254:36595/ using document root <none> and password file <none>
I20260812 14:46:40.761221 25594 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:40.761270 25594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:40.761394 25594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:40.762379 25594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "82eb3d8a369646febd1e8b737e96cc2c"
format_stamp: "Formatted at 2026-08-12 14:46:40 on dist-test-slave-q9h9"
server_key: "67b768c9f638c9efca0e2dae4f063419"
server_key_iv: "fb41ff2234fe2b926d016c705e7e352a"
server_key_version: "encryptionkey@0"
I20260812 14:46:40.762682 25594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "82eb3d8a369646febd1e8b737e96cc2c"
format_stamp: "Formatted at 2026-08-12 14:46:40 on dist-test-slave-q9h9"
server_key: "67b768c9f638c9efca0e2dae4f063419"
server_key_iv: "fb41ff2234fe2b926d016c705e7e352a"
server_key_version: "encryptionkey@0"
I20260812 14:46:40.763846 25594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:40.764498 25630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:40.764665 25594 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:40.764756 25594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "82eb3d8a369646febd1e8b737e96cc2c"
format_stamp: "Formatted at 2026-08-12 14:46:40 on dist-test-slave-q9h9"
server_key: "67b768c9f638c9efca0e2dae4f063419"
server_key_iv: "fb41ff2234fe2b926d016c705e7e352a"
server_key_version: "encryptionkey@0"
I20260812 14:46:40.764815 25594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:40.767877 25594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:40.769460 25594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:40.769583 25594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:40.773288 25594 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:37203
I20260812 14:46:40.773324 25717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:37203 every 8 connection(s)
I20260812 14:46:40.773586 25594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:40.774273 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25594
I20260812 14:46:40.774389 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:40.774534 25720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:40.774554 16271 external_mini_cluster.cc:1428] Setting key 4d9d42e3dc12e3c5e0240784652c1e33
I20260812 14:46:40.776598 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:40.774988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57005 (local address 127.15.227.254:37203)
0812 14:46:40.775170 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:40.775179 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:40.775213 (+ 34us) server_negotiation.cc:407] Connection header received
0812 14:46:40.775558 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:40.775566 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:40.775596 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:40.775748 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:40.776226 (+ 478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:40.776230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:40.776231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:40.776235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:40.776280 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:40.776293 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:40.776364 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:40.776396 (+ 32us) server_negotiation.cc:299] Negotiation successful
0812 14:46:40.776432 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260812 14:46:40.777491 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:40.777522 25720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c: Bootstrap starting.
I20260812 14:46:40.778088 25720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:40.778373 25720 log.cc:824] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:40.779273 25720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c: No bootstrap required, opened a new log
I20260812 14:46:40.779600 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:40.778736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37293 (local address 127.15.227.254:37203)
0812 14:46:40.778770 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:40.778771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:40.778897 (+ 126us) server_negotiation.cc:407] Connection header received
0812 14:46:40.778921 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:40.778922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:40.778926 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:40.778982 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:40.779432 (+ 450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:40.779433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:40.779434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:40.779437 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:40.779448 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:40.779451 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:40.779491 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:40.779541 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 14:46:40.779550 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260812 14:46:40.781087 25720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eb3d8a369646febd1e8b737e96cc2c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37203 } }
I20260812 14:46:40.781208 25720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:40.781242 25720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82eb3d8a369646febd1e8b737e96cc2c, State: Initialized, Role: FOLLOWER
I20260812 14:46:40.781353 25720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eb3d8a369646febd1e8b737e96cc2c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37203 } }
I20260812 14:46:40.781436 25720 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:40.781467 25720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:40.781513 25720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:40.782001 25720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eb3d8a369646febd1e8b737e96cc2c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37203 } }
I20260812 14:46:40.782177 25720 leader_election.cc:302] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82eb3d8a369646febd1e8b737e96cc2c; no voters:
I20260812 14:46:40.782330 25720 leader_election.cc:288] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:40.782377 25734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:40.782539 25734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [term 1 LEADER]: Becoming Leader. State: Replica: 82eb3d8a369646febd1e8b737e96cc2c, State: Running, Role: LEADER
I20260812 14:46:40.782549 25720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:40.782687 25734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eb3d8a369646febd1e8b737e96cc2c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37203 } }
I20260812 14:46:40.783031 25738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82eb3d8a369646febd1e8b737e96cc2c. Latest consensus state: current_term: 1 leader_uuid: "82eb3d8a369646febd1e8b737e96cc2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eb3d8a369646febd1e8b737e96cc2c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37203 } } }
I20260812 14:46:40.783015 25737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82eb3d8a369646febd1e8b737e96cc2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82eb3d8a369646febd1e8b737e96cc2c" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 37203 } } }
I20260812 14:46:40.783135 25738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:40.783139 25737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:40.783608 25750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:40.784058 25750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:40.785385 25750 catalog_manager.cc:1347] Generated new cluster ID: 57631d8a13eb460a9d3f105aa620c98e
I20260812 14:46:40.785434 25750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:40.792915 25750 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:40.793047 25750 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:40.798529 25750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82eb3d8a369646febd1e8b737e96cc2c: Generated new TSK 0
I20260812 14:46:40.798668 25750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:40.800863 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25594
2026-08-12T14:46:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:40 dist-test-slave-q9h9 krb5kdc[25773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:40 dist-test-slave-q9h9 krb5kdc[25773](info): set up 2 sockets
Aug 12 14:46:40 dist-test-slave-q9h9 krb5kdc[25773](info): commencing operation
krb5kdc: starting...
W20260812 14:46:41.981889 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:42 dist-test-slave-q9h9 krb5kdc[25773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:42.092137 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:39437
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:43211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:39437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:42.170632 25799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:42.170814 25799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:42.170845 25799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:42.171938 25799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:42.171980 25799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:42.171999 25799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:42.172015 25799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:42.172031 25799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:42.173460 25799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:43211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:39437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:39437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:42.173681 25799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:42.173877 25799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:42.178328 25810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:42.178340 25811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:42.178377 25816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:42.178503 25799 server_base.cc:1034] running on GCE node
I20260812 14:46:42.178702 25799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:42.178972 25799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:42.180109 25799 hybrid_clock.cc:648] HybridClock initialized: now 1786546002180091 us; error 34 us; skew 500 ppm
Aug 12 14:46:42 dist-test-slave-q9h9 krb5kdc[25773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546002, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:42.182274 25799 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:42.182649 25799 webserver.cc:460] Webserver started at http://127.15.227.254:38651/ using document root <none> and password file <none>
I20260812 14:46:42.182840 25799 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:42.182885 25799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:42.183005 25799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:42.183947 25799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "fcda6ed9a03c4d55b8a70d552e78a866"
format_stamp: "Formatted at 2026-08-12 14:46:42 on dist-test-slave-q9h9"
server_key: "ed26b953a687141b2588f94a100f1655"
server_key_iv: "52f7625bb8628b56bb6336a017f4303c"
server_key_version: "encryptionkey@0"
I20260812 14:46:42.184258 25799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "fcda6ed9a03c4d55b8a70d552e78a866"
format_stamp: "Formatted at 2026-08-12 14:46:42 on dist-test-slave-q9h9"
server_key: "ed26b953a687141b2588f94a100f1655"
server_key_iv: "52f7625bb8628b56bb6336a017f4303c"
server_key_version: "encryptionkey@0"
I20260812 14:46:42.185426 25799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260812 14:46:42.186110 25828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:42.186252 25799 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:42.186343 25799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "fcda6ed9a03c4d55b8a70d552e78a866"
format_stamp: "Formatted at 2026-08-12 14:46:42 on dist-test-slave-q9h9"
server_key: "ed26b953a687141b2588f94a100f1655"
server_key_iv: "52f7625bb8628b56bb6336a017f4303c"
server_key_version: "encryptionkey@0"
I20260812 14:46:42.186410 25799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:42.192893 25799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:42.194418 25799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:42.194536 25799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:42.198171 25799 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:39437
I20260812 14:46:42.198200 25942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:39437 every 8 connection(s)
I20260812 14:46:42.198480 25799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:42.199237 25943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:42.201189 25943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866: Bootstrap starting.
I20260812 14:46:42.201709 25943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:42.201961 25943 log.cc:824] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:42.202602 25943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866: No bootstrap required, opened a new log
I20260812 14:46:42.204107 25943 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39437 } }
I20260812 14:46:42.204231 25943 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:42.204260 25943 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcda6ed9a03c4d55b8a70d552e78a866, State: Initialized, Role: FOLLOWER
I20260812 14:46:42.204385 25943 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39437 } }
I20260812 14:46:42.204447 25943 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:42.204476 25943 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:42.204520 25943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:42.205284 25943 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39437 } }
I20260812 14:46:42.205379 25943 leader_election.cc:302] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcda6ed9a03c4d55b8a70d552e78a866; no voters:
I20260812 14:46:42.205529 25943 leader_election.cc:288] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:42.205556 25950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:42.205693 25950 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [term 1 LEADER]: Becoming Leader. State: Replica: fcda6ed9a03c4d55b8a70d552e78a866, State: Running, Role: LEADER
I20260812 14:46:42.205761 25943 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:42.205781 25950 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39437 } }
I20260812 14:46:42.205861 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 25799
I20260812 14:46:42.205979 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:42.206195 16271 external_mini_cluster.cc:1428] Setting key c70c93798cad3e310fa2d3603a253c7f
I20260812 14:46:42.206224 25950 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcda6ed9a03c4d55b8a70d552e78a866. Latest consensus state: current_term: 1 leader_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39437 } } }
I20260812 14:46:42.206295 25950 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:42.206445 25952 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcda6ed9a03c4d55b8a70d552e78a866" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 39437 } } }
I20260812 14:46:42.206511 25952 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:42.207020 25966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:42.207512 25966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:42.209268 25966 catalog_manager.cc:1347] Generated new cluster ID: e9aa64afa05340b4bc57aa96ac4268e5
I20260812 14:46:42.209311 25966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 12 14:46:42 dist-test-slave-q9h9 krb5kdc[25773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:42.217445 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:42.206778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52853 (local address 127.15.227.254:39437)
0812 14:46:42.207030 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:42.207038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:42.207069 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:46:42.207192 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:42.207199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:42.207253 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:42.207389 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:42.208123 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:42.209168 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:42.210283 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:42.210874 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:42.210966 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:42.213508 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:42.213512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:42.213512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:42.213524 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:42.215956 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:42.216311 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:42.216313 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:42.216314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:42.216428 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:42.216600 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:42.216601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:42.216602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:42.216895 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:42.217040 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:42.217116 (+ 76us) server_negotiation.cc:299] Negotiation successful
0812 14:46:42.217244 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
I20260812 14:46:42.218302 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:42.220745 25966 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:42.221395 25966 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:42.225203 25984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:42.219765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39735 (local address 127.15.227.254:39437)
0812 14:46:42.219831 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:42.219832 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:42.219840 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:42.219923 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:42.219924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:42.219935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:42.219987 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:42.220585 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:42.221117 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:42.221818 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:42.222042 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:42.222100 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:42.223066 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:42.223068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:42.223068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:42.223074 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:42.224245 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:42.224548 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:42.224549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:42.224550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:42.224600 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:42.224763 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:42.224764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:42.224764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:42.224859 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:42.224950 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:42.225040 (+ 90us) server_negotiation.cc:299] Negotiation successful
0812 14:46:42.225112 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 14:46:42.230528 25966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcda6ed9a03c4d55b8a70d552e78a866: Generated new TSK 0
I20260812 14:46:42.230677 25966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:42.234633 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 25799
2026-08-12T14:46:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:42 dist-test-slave-q9h9 krb5kdc[26013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:42 dist-test-slave-q9h9 krb5kdc[26013](info): set up 2 sockets
Aug 12 14:46:42 dist-test-slave-q9h9 krb5kdc[26013](info): commencing operation
krb5kdc: starting...
W20260812 14:46:43.429914 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:43 dist-test-slave-q9h9 krb5kdc[26013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:43.536291 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34395
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:42141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:34395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:43.613620 26035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:43.613796 26035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:43.613826 26035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:43.614959 26035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:43.615001 26035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:43.615020 26035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:43.615036 26035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:43.615052 26035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:43.616497 26035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:42141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:34395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:34395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:43.616722 26035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:43.616918 26035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:43.621243 26044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:43.621311 26050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:43.621325 26048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:43.621460 26035 server_base.cc:1034] running on GCE node
I20260812 14:46:43.621670 26035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:43.621901 26035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:43.623045 26035 hybrid_clock.cc:648] HybridClock initialized: now 1786546003623027 us; error 31 us; skew 500 ppm
Aug 12 14:46:43 dist-test-slave-q9h9 krb5kdc[26013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546003, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:43.625278 26035 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:43.625715 26035 webserver.cc:460] Webserver started at http://127.15.227.254:34797/ using document root <none> and password file <none>
I20260812 14:46:43.625968 26035 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:43.626049 26035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:43.626190 26035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:43.627135 26035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "01e58c3695b34cb59b83765a570c0c69"
format_stamp: "Formatted at 2026-08-12 14:46:43 on dist-test-slave-q9h9"
server_key: "8966ddc1a1a64d836ff26aed071059d4"
server_key_iv: "b9089266cfc5175c434cf2e91dbebe81"
server_key_version: "encryptionkey@0"
I20260812 14:46:43.627434 26035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "01e58c3695b34cb59b83765a570c0c69"
format_stamp: "Formatted at 2026-08-12 14:46:43 on dist-test-slave-q9h9"
server_key: "8966ddc1a1a64d836ff26aed071059d4"
server_key_iv: "b9089266cfc5175c434cf2e91dbebe81"
server_key_version: "encryptionkey@0"
I20260812 14:46:43.628683 26035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:43.630010 26063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:43.630206 26035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:43.630300 26035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "01e58c3695b34cb59b83765a570c0c69"
format_stamp: "Formatted at 2026-08-12 14:46:43 on dist-test-slave-q9h9"
server_key: "8966ddc1a1a64d836ff26aed071059d4"
server_key_iv: "b9089266cfc5175c434cf2e91dbebe81"
server_key_version: "encryptionkey@0"
I20260812 14:46:43.630373 26035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:43.647086 26035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:43.648954 26035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:43.649053 26035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:43.652751 26035 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:34395
I20260812 14:46:43.652786 26159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:34395 every 8 connection(s)
I20260812 14:46:43.653071 26035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:43.653834 26164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:43.655881 26164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69: Bootstrap starting.
I20260812 14:46:43.656415 26164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:43.656667 26164 log.cc:824] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:43.657285 26164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69: No bootstrap required, opened a new log
I20260812 14:46:43.658829 26164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e58c3695b34cb59b83765a570c0c69" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34395 } }
I20260812 14:46:43.658953 26164 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:43.658974 26164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e58c3695b34cb59b83765a570c0c69, State: Initialized, Role: FOLLOWER
I20260812 14:46:43.659103 26164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e58c3695b34cb59b83765a570c0c69" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34395 } }
I20260812 14:46:43.659164 26164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:43.659184 26164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:43.659211 26164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:43.659977 26164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e58c3695b34cb59b83765a570c0c69" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34395 } }
I20260812 14:46:43.660044 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26035
I20260812 14:46:43.660074 26164 leader_election.cc:302] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e58c3695b34cb59b83765a570c0c69; no voters:
I20260812 14:46:43.660148 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:43.660231 26164 leader_election.cc:288] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:43.660275 26171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:43.660334 16271 external_mini_cluster.cc:1428] Setting key a34cf7eb8b8c67a945d840c72d3a73fe
I20260812 14:46:43.660389 26171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [term 1 LEADER]: Becoming Leader. State: Replica: 01e58c3695b34cb59b83765a570c0c69, State: Running, Role: LEADER
I20260812 14:46:43.660485 26164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:43.660495 26171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e58c3695b34cb59b83765a570c0c69" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34395 } }
I20260812 14:46:43.660887 26171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e58c3695b34cb59b83765a570c0c69. Latest consensus state: current_term: 1 leader_uuid: "01e58c3695b34cb59b83765a570c0c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e58c3695b34cb59b83765a570c0c69" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34395 } } }
I20260812 14:46:43.660959 26171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:43.660925 26172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e58c3695b34cb59b83765a570c0c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e58c3695b34cb59b83765a570c0c69" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 34395 } } }
I20260812 14:46:43.661005 26172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:43.661199 26192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:43.661718 26192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:43.663745 26192 catalog_manager.cc:1347] Generated new cluster ID: 667b22fb8ef94c72a75f337d932ae4d5
I20260812 14:46:43.663800 26192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:43.666673 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:43.660912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54889 (local address 127.15.227.254:34395)
0812 14:46:43.661888 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:43.661896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:43.661927 (+ 31us) server_negotiation.cc:407] Connection header received
0812 14:46:43.662053 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:43.662060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:43.662117 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:43.662278 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:43.662508 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:43.663626 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:43.664702 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:43.665304 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:43.665422 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:43.665976 (+ 554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:43.665980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:43.665980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:43.666203 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:43.666241 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:43.666256 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:43.666326 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:43.666396 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:46:43.666458 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":87,"threads_started":1}
I20260812 14:46:43.667474 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:43.668702 26192 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:43.669327 26192 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:43.671694 26192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e58c3695b34cb59b83765a570c0c69: Generated new TSK 0
I20260812 14:46:43.671857 26192 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:43.672539 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:43.668923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53175 (local address 127.15.227.254:34395)
0812 14:46:43.668958 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:43.668959 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:43.669146 (+ 187us) server_negotiation.cc:407] Connection header received
0812 14:46:43.669190 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:43.669191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:43.669340 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:43.669395 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:43.669572 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:43.670338 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:43.671425 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:43.671668 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:43.671735 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:43.672342 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:43.672343 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:43.672344 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:43.672348 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:43.672360 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:43.672363 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:43.672407 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:43.672454 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 14:46:43.672476 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260812 14:46:43.674271 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26035
2026-08-12T14:46:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:43 dist-test-slave-q9h9 krb5kdc[26225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:43 dist-test-slave-q9h9 krb5kdc[26225](info): set up 2 sockets
Aug 12 14:46:43 dist-test-slave-q9h9 krb5kdc[26225](info): commencing operation
krb5kdc: starting...
W20260812 14:46:44.867872 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:44 dist-test-slave-q9h9 krb5kdc[26225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:44.974102 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43845
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:44475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:43845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:45.054198 26243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:45.054374 26243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:45.054395 26243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:45.055558 26243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:45.055605 26243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:45.055619 26243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:45.055641 26243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:45.055673 26243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:45.057312 26243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:44475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:43845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:43845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:45.057574 26243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:45.057780 26243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:46:45.062316 26243 server_base.cc:1034] running on GCE node
W20260812 14:46:45.062260 26253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:45.062284 26249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:45.062276 26250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:45.062703 26243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:45.062978 26243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:45.064114 26243 hybrid_clock.cc:648] HybridClock initialized: now 1786546005064103 us; error 25 us; skew 500 ppm
Aug 12 14:46:45 dist-test-slave-q9h9 krb5kdc[26225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546005, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:45.069356 26243 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:45.069772 26243 webserver.cc:460] Webserver started at http://127.15.227.254:40481/ using document root <none> and password file <none>
I20260812 14:46:45.069983 26243 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:45.070057 26243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:45.070189 26243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:45.071151 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "36b7e1226d7347c28dc957b85e60dc90"
format_stamp: "Formatted at 2026-08-12 14:46:45 on dist-test-slave-q9h9"
server_key: "79f77596eabe5e2e98dd87d6e7edbbe3"
server_key_iv: "ba7f57733eaa5cefb3306fe47836b9f2"
server_key_version: "encryptionkey@0"
I20260812 14:46:45.071472 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "36b7e1226d7347c28dc957b85e60dc90"
format_stamp: "Formatted at 2026-08-12 14:46:45 on dist-test-slave-q9h9"
server_key: "79f77596eabe5e2e98dd87d6e7edbbe3"
server_key_iv: "ba7f57733eaa5cefb3306fe47836b9f2"
server_key_version: "encryptionkey@0"
I20260812 14:46:45.072654 26243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:45.073279 26266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:45.073422 26243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:45.073504 26243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "36b7e1226d7347c28dc957b85e60dc90"
format_stamp: "Formatted at 2026-08-12 14:46:45 on dist-test-slave-q9h9"
server_key: "79f77596eabe5e2e98dd87d6e7edbbe3"
server_key_iv: "ba7f57733eaa5cefb3306fe47836b9f2"
server_key_version: "encryptionkey@0"
I20260812 14:46:45.073570 26243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:45.076689 26243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:45.078665 26243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:45.078788 26243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:45.082510 26243 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:43845
I20260812 14:46:45.082541 26359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:43845 every 8 connection(s)
I20260812 14:46:45.082819 26243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:45.083730 26361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:45.085820 26361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90: Bootstrap starting.
I20260812 14:46:45.086423 26361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:45.086671 26361 log.cc:824] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:45.087373 26361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90: No bootstrap required, opened a new log
I20260812 14:46:45.087956 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26243
I20260812 14:46:45.088093 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:45.088287 16271 external_mini_cluster.cc:1428] Setting key 53dd5fbcc0947404b2f7adfccdc791c9
I20260812 14:46:45.089814 26361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b7e1226d7347c28dc957b85e60dc90" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43845 } }
I20260812 14:46:45.089967 26361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 14:46:45.090001 26361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b7e1226d7347c28dc957b85e60dc90, State: Initialized, Role: FOLLOWER
I20260812 14:46:45.090152 26361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b7e1226d7347c28dc957b85e60dc90" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43845 } }
I20260812 14:46:45.090267 26361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 14:46:45.090310 26361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 14:46:45.090359 26361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:45.090859 26361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b7e1226d7347c28dc957b85e60dc90" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43845 } }
I20260812 14:46:45.090925 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:45.088761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36931 (local address 127.15.227.254:43845)
0812 14:46:45.088908 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:45.088915 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:45.088938 (+ 23us) server_negotiation.cc:407] Connection header received
0812 14:46:45.089757 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:45.089764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:45.089790 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:45.089897 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:45.090534 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:45.090538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:45.090538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:45.090542 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:45.090579 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:45.090596 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:45.090653 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:45.090714 (+ 61us) server_negotiation.cc:299] Negotiation successful
0812 14:46:45.090752 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260812 14:46:45.090952 26361 leader_election.cc:302] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36b7e1226d7347c28dc957b85e60dc90; no voters:
I20260812 14:46:45.091097 26361 leader_election.cc:288] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 14:46:45.091320 26370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:45.091343 26361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:45.091498 26370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [term 1 LEADER]: Becoming Leader. State: Replica: 36b7e1226d7347c28dc957b85e60dc90, State: Running, Role: LEADER
I20260812 14:46:45.091598 26370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b7e1226d7347c28dc957b85e60dc90" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43845 } }
I20260812 14:46:45.091789 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:45.092077 26373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36b7e1226d7347c28dc957b85e60dc90. Latest consensus state: current_term: 1 leader_uuid: "36b7e1226d7347c28dc957b85e60dc90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b7e1226d7347c28dc957b85e60dc90" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43845 } } }
I20260812 14:46:45.092157 26373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:45.092360 26387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:45.092751 26371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36b7e1226d7347c28dc957b85e60dc90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36b7e1226d7347c28dc957b85e60dc90" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 43845 } } }
I20260812 14:46:45.092828 26371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:45.093007 26387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:45.094511 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:45.093464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57349 (local address 127.15.227.254:43845)
0812 14:46:45.093495 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:45.093497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:45.093625 (+ 128us) server_negotiation.cc:407] Connection header received
0812 14:46:45.093675 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:45.093676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:45.093681 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:45.093754 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:45.094328 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:45.094329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:45.094329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 14:46:45.094333 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:45.094345 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 14:46:45.094349 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:45.094392 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:45.094442 (+ 50us) server_negotiation.cc:299] Negotiation successful
0812 14:46:45.094455 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:46:45.094558 26387 catalog_manager.cc:1347] Generated new cluster ID: 37ce1522b1434a878f1fc4d56c47722f
I20260812 14:46:45.094601 26387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:45.103459 26387 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:45.103605 26387 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:45.115368 26387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36b7e1226d7347c28dc957b85e60dc90: Generated new TSK 0
I20260812 14:46:45.115523 26387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 14:46:45.119658 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26243
2026-08-12T14:46:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1440 ms)
[----------] 18 tests from AuthTokenIssuingTest (26313 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:45 dist-test-slave-q9h9 krb5kdc[26429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:45 dist-test-slave-q9h9 krb5kdc[26429](info): set up 2 sockets
Aug 12 14:46:45 dist-test-slave-q9h9 krb5kdc[26429](info): commencing operation
krb5kdc: starting...
W20260812 14:46:46.318193 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:46.422922 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:41209
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:41209,127.15.227.253:45547,127.15.227.252:35471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:46.500509 26452 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:46:46.500702 26452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:46.500728 26452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:46.500752 26452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:46.501876 26452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:46.501916 26452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:46.501935 26452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:46.501951 26452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:46.501967 26452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:46.503433 26452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:41209,127.15.227.253:45547,127.15.227.252:35471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:41209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:46.503667 26452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:46.503859 26452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:46.508296 26461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:46.508315 26460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:46.508364 26452 server_base.cc:1034] running on GCE node
W20260812 14:46:46.508297 26464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:46.508793 26452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:46.509044 26452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:46.510185 26452 hybrid_clock.cc:648] HybridClock initialized: now 1786546006510166 us; error 33 us; skew 500 ppm
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:46.512393 26452 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:46.512784 26452 webserver.cc:460] Webserver started at http://127.15.227.254:45681/ using document root <none> and password file <none>
I20260812 14:46:46.512979 26452 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:46.513024 26452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:46.513139 26452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:46.514113 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "dea8ba3f45f14666a460263a0e3a22d9"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "cf11955998f08a7224ebd4a4db00cc26"
server_key_iv: "9afe300f10b33c482dbb2164a790c86b"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.514427 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "dea8ba3f45f14666a460263a0e3a22d9"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "cf11955998f08a7224ebd4a4db00cc26"
server_key_iv: "9afe300f10b33c482dbb2164a790c86b"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.515599 26452 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:46.516238 26479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:46.516386 26452 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:46.516466 26452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "dea8ba3f45f14666a460263a0e3a22d9"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "cf11955998f08a7224ebd4a4db00cc26"
server_key_iv: "9afe300f10b33c482dbb2164a790c86b"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.516525 26452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:46.529693 26452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:46.531298 26452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:46.531424 26452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:46.535178 26452 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:41209
I20260812 14:46:46.535209 26574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:41209 every 8 connection(s)
I20260812 14:46:46.535498 26452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:46.536279 26575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:46.536536 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26452
I20260812 14:46:46.536645 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:46.536855 16271 external_mini_cluster.cc:1428] Setting key e53bbf73b2daa0580ec1fe8ef12ae60c
I20260812 14:46:46.538062 26575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.227.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.253@KRBTEST.COM" created.
I20260812 14:46:46.547330 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.538756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41209 (local address 127.0.0.1:36688)
0812 14:46:46.538919 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.538947 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.539337 (+ 390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.539637 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.539640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.539933 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.540131 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.540136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.541149 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.541150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.542204 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.542207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.542637 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.542638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.542726 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.543141 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.543150 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.544548 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.546059 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.546066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.546068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.546257 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.546456 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.546457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.546457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.546535 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.546838 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.546843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.546901 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.546977 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.547183 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":81,"thread_start_us":84,"threads_started":1}
I20260812 14:46:46.547338 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.538773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36688 (local address 127.15.227.254:41209)
0812 14:46:46.539453 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.539456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.539466 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:46:46.539516 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.539518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.539531 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.539586 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.540217 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.541051 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.542277 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.542560 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.542631 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.544622 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.544629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.544630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.544635 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.545969 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.546326 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.546329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.546330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.546401 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.546588 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.546589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.546590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.546778 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.546843 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.547018 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.547118 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":82,"threads_started":1}
I20260812 14:46:46.548126 26575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:46.548416 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.548319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:45547 (local address 127.0.0.1:51064)
0812 14:46:46.548355 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.548396 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 14:46:46.548512 26482 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.253:45547: connect: Connection refused (error 111)
I20260812 14:46:46.548900 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.548813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:45547 (local address 127.0.0.1:51076)
0812 14:46:46.548854 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.548878 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260812 14:46:46.548993 26575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:45547: Network error: Client connection negotiation failed: client connection to 127.15.227.253:45547: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab.
Entry for principal kudu/127.15.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab.
I20260812 14:46:46.558337 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:45547
--webserver_interface=127.15.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab
--principal=kudu/127.15.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:41209,127.15.227.253:45547,127.15.227.252:35471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:46:46.598187 26575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } attempt: 1
I20260812 14:46:46.598515 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.598420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:45547 (local address 127.0.0.1:51086)
0812 14:46:46.598469 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.598494 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:46:46.598829 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.598714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:45547 (local address 127.0.0.1:51096)
0812 14:46:46.598765 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.598802 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:45547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:46:46.598924 26575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:45547: Network error: Client connection negotiation failed: client connection to 127.15.227.253:45547: connect: Connection refused (error 111)
W20260812 14:46:46.636921 26591 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:46:46.637092 26591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:46.637107 26591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:46.637125 26591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:46.638219 26591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:46.638252 26591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:46.638264 26591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:46.638275 26591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:46.638293 26591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:46.639678 26591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:41209,127.15.227.253:45547,127.15.227.252:35471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab
--principal=kudu/127.15.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:45547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:46.639851 26591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:46.640013 26591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:46.644388 26597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:46.644368 26599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:46.644361 26603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:46.644459 26591 server_base.cc:1034] running on GCE node
I20260812 14:46:46.644754 26591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:46.645012 26591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:46.646145 26591 hybrid_clock.cc:648] HybridClock initialized: now 1786546006646136 us; error 32 us; skew 500 ppm
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:46.648360 26591 init.cc:377] Logged in from keytab as kudu/127.15.227.253@KRBTEST.COM (short username kudu)
I20260812 14:46:46.648711 26591 webserver.cc:460] Webserver started at http://127.15.227.253:37019/ using document root <none> and password file <none>
I20260812 14:46:46.648887 26591 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:46.648923 26591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:46.649008 26591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:46.649904 26591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data/instance:
uuid: "085dff5bac7749b78f8b0a0566e864e9"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "cfddf0fd43df6e6c8e6add83c1103367"
server_key_iv: "80399bda160a841bb0f5658a1a5a201e"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.650223 26591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/wal/instance:
uuid: "085dff5bac7749b78f8b0a0566e864e9"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "cfddf0fd43df6e6c8e6add83c1103367"
server_key_iv: "80399bda160a841bb0f5658a1a5a201e"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.651337 26591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:46.651963 26613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:46.652101 26591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:46.652180 26591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/wal
uuid: "085dff5bac7749b78f8b0a0566e864e9"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "cfddf0fd43df6e6c8e6add83c1103367"
server_key_iv: "80399bda160a841bb0f5658a1a5a201e"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.652235 26591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:46.661018 26591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:46.662616 26591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:46.662721 26591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:46.666302 26591 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.253:45547
I20260812 14:46:46.666334 26711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.253:45547 every 8 connection(s)
I20260812 14:46:46.666622 26591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
I20260812 14:46:46.667047 26575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } attempt: 2
I20260812 14:46:46.667480 26712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:46.669063 26712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:46.672155 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26591
I20260812 14:46:46.672286 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-1/wal/instance
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:46:46.672473 16271 external_mini_cluster.cc:1428] Setting key e5f7dad769f54446a440f7a9eb3a194d
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:46.676363 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.667305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:45547 (local address 127.0.0.1:51104)
0812 14:46:46.667357 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.667367 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.667414 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.668196 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.668198 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.668350 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.668391 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.668393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.669465 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.669467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.670230 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.670233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.670910 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.670911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.670960 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.671329 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.671334 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.672704 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.675160 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.675161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.675162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.675404 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.675653 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.675654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.675654 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.675719 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.676073 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.676074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.676118 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.676183 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.676275 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:46:46.676596 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.667429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.15.227.253:45547)
0812 14:46:46.667632 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.667641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.667666 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:46:46.668030 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.668036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.668076 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.668192 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.668480 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.669391 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.670300 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.670826 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.670931 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.672990 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.672997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.672997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.673008 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.674864 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.675473 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.675476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.675477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.675577 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.675790 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.675791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.675791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.676006 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.676076 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.676255 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.676407 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260812 14:46:46.677290 26575 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:46.677439 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.669617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.15.227.254:41209)
0812 14:46:46.669664 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.669666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.669814 (+ 148us) server_negotiation.cc:407] Connection header received
0812 14:46:46.669871 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.669872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.669882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.669927 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.670472 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.671163 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.672129 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.672404 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.672463 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.674719 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.674720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.674721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.674728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.676177 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.676617 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.676618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.676618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.676686 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.676884 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.676886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.676886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.677021 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.677103 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.677283 (+ 180us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.677362 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:46:46.677618 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.677512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55216)
0812 14:46:46.677571 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.677593 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 14:46:46.677706 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.669560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41209 (local address 127.0.0.1:36700)
0812 14:46:46.669714 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.669755 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.669821 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.669943 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.669945 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.670322 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.670401 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.670403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.671263 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.671264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.672060 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.672063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.672524 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.672526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.672603 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.673009 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.673014 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.674629 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.676267 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.676274 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.676276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.676547 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.676769 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.676770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.676771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.676834 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.677097 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.677101 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.677149 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.677501 (+ 352us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.677616 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260812 14:46:46.677944 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.677868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55226)
0812 14:46:46.677897 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.677924 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260812 14:46:46.678128 26575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:35471: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
I20260812 14:46:46.678228 26712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.227.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.252@KRBTEST.COM" created.
I20260812 14:46:46.683660 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.678421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:45547 (local address 127.0.0.1:51116)
0812 14:46:46.678458 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.678468 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.678504 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.678603 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.678604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.678726 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.678751 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.678752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.679325 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.679326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.679775 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.679776 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.680085 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.680085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.680130 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.680450 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.680454 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.681466 (+ 1012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.682847 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.682847 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.682848 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.682992 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.683146 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.683147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.683147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.683204 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.683394 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.683395 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.683429 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.683488 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.683569 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260812 14:46:46.684757 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.678468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.15.227.253:45547)
0812 14:46:46.678500 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.678502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.678509 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:46.678553 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.678554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.678561 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.678602 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.678810 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.679261 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.679836 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.680032 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.680082 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.681534 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.681535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.681535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.681541 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.682776 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.683049 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.683050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.683050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.683097 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.683251 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.683252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.683252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.683344 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.683396 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.683554 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.683623 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":18}
I20260812 14:46:46.685067 26712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:46.685355 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.685254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55232)
0812 14:46:46.685290 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.685333 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260812 14:46:46.685472 26617 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
I20260812 14:46:46.685813 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.685736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55248)
0812 14:46:46.685769 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.685792 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 14:46:46.685940 26712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:35471: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab.
Entry for principal kudu/127.15.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab.
I20260812 14:46:46.694645 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:35471
--webserver_interface=127.15.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab
--principal=kudu/127.15.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:36617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:41209,127.15.227.253:45547,127.15.227.252:35471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:46:46.721252 26575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } attempt: 1
I20260812 14:46:46.721572 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.721447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55254)
0812 14:46:46.721501 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.721541 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260812 14:46:46.721853 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.721774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55256)
0812 14:46:46.721810 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.721834 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:46:46.721951 26575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:35471: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
I20260812 14:46:46.735102 26712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } attempt: 1
I20260812 14:46:46.735412 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.735301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55260)
0812 14:46:46.735348 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.735384 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:46:46.735728 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.735635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55266)
0812 14:46:46.735681 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.735707 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 14:46:46.735821 26712 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:35471: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
I20260812 14:46:46.769078 26575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } attempt: 2
I20260812 14:46:46.769428 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.769304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55282)
0812 14:46:46.769360 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.769399 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:46:46.769762 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.769659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55294)
0812 14:46:46.769703 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.769741 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260812 14:46:46.769851 26575 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:35471: Network error: Client connection negotiation failed: client connection to 127.15.227.252:35471: connect: Connection refused (error 111)
W20260812 14:46:46.773005 26726 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:46:46.773172 26726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:46.773187 26726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:46.773211 26726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:46.774322 26726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:46.774359 26726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:46.774371 26726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:46.774389 26726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:46.774401 26726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:46.775802 26726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:36617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:41209,127.15.227.253:45547,127.15.227.252:35471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab
--principal=kudu/127.15.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:35471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:46.775967 26726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:46.776129 26726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:46.780434 26733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:46.780516 26726 server_base.cc:1034] running on GCE node
W20260812 14:46:46.780427 26736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:46.780434 26738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:46.780786 26726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:46.781052 26726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:46.782184 26726 hybrid_clock.cc:648] HybridClock initialized: now 1786546006782169 us; error 26 us; skew 500 ppm
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:46.784420 26726 init.cc:377] Logged in from keytab as kudu/127.15.227.252@KRBTEST.COM (short username kudu)
I20260812 14:46:46.784811 26726 webserver.cc:460] Webserver started at http://127.15.227.252:44479/ using document root <none> and password file <none>
I20260812 14:46:46.785015 26726 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:46.785061 26726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:46.785179 26726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:46.786166 26726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data/instance:
uuid: "5ef9fc6efb5544eabb6ebafece3279d2"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "f3603450876b8b684dffff30f495c147"
server_key_iv: "260d835537a2a686ac00a36fccda739a"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.786480 26726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/wal/instance:
uuid: "5ef9fc6efb5544eabb6ebafece3279d2"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "f3603450876b8b684dffff30f495c147"
server_key_iv: "260d835537a2a686ac00a36fccda739a"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.787623 26726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:46.788266 26746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:46.788404 26726 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:46.788482 26726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/wal
uuid: "5ef9fc6efb5544eabb6ebafece3279d2"
format_stamp: "Formatted at 2026-08-12 14:46:46 on dist-test-slave-q9h9"
server_key: "f3603450876b8b684dffff30f495c147"
server_key_iv: "260d835537a2a686ac00a36fccda739a"
server_key_version: "encryptionkey@0"
I20260812 14:46:46.788540 26726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:46.797473 26726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:46.799386 26726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:46.799508 26726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:46.803133 26726 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.252:35471
I20260812 14:46:46.803160 26834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.252:35471 every 8 connection(s)
I20260812 14:46:46.803447 26726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
I20260812 14:46:46.803948 26712 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } attempt: 2
I20260812 14:46:46.804276 26835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:46.806041 26835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:46.808393 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 26726
I20260812 14:46:46.808486 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786545740023890-16271-0/minicluster-data/master-2/wal/instance
I20260812 14:46:46.808672 16271 external_mini_cluster.cc:1428] Setting key d94a1e7aad41a14267d5d51adebfeb6d
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:46.812810 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.804191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55304)
0812 14:46:46.804249 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.804261 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.804312 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.805036 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.805037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.805199 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.805247 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.805250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.806298 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.806300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.806929 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.806932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.807457 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.807458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.807513 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.807858 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.807864 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.809157 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.811497 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.811498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.811499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.811717 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.811964 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.811965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.811965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.812033 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.812499 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.812500 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.812549 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.812613 (+ 64us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.812705 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 14:46:46.813278 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.804304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.15.227.252:35471)
0812 14:46:46.804477 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.804485 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.804512 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:46:46.804873 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.804879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.804920 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.805019 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.805333 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.806219 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.807006 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.807387 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.807478 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.809245 (+ 1767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.809253 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.809254 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.809265 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.811410 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.811799 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.811802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.811802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.811895 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.812102 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.812103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.812103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.812391 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.812467 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.812963 (+ 496us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.813085 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260812 14:46:46.813802 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.806642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.15.227.254:41209)
0812 14:46:46.806696 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.806698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.806819 (+ 121us) server_negotiation.cc:407] Connection header received
0812 14:46:46.806880 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.806881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.806890 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.806935 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.807337 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.807780 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.808590 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.808884 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.808951 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.811125 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.811127 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.811127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.811134 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.812426 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.812825 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.812827 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.812827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.812889 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.813091 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.813092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.813092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.813229 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.813271 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.813666 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.813745 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:46:46.814067 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.806589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:41209 (local address 127.0.0.1:36710)
0812 14:46:46.806731 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.806763 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.806823 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.806951 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.806953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.807197 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.807260 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.807262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.807855 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.807856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.808514 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.808516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.808963 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.808964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.809042 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.809492 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.809497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.811053 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.812498 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.812504 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.812506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.812740 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.812963 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.812964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.812964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.813021 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.813321 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.813328 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.813528 (+ 200us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.813863 (+ 335us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.813972 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260812 14:46:46.814579 26835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:46.815050 26712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: Bootstrap starting.
I20260812 14:46:46.816008 26712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:46.816326 26712 log.cc:824] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:46.817077 26712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: No bootstrap required, opened a new log
I20260812 14:46:46.817513 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.809166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.15.227.252:35471)
0812 14:46:46.809290 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.809292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.809299 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:46.809536 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.809537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.809547 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.809612 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.810364 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.810874 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.811871 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.812084 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.812146 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.814527 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.814528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.814529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.814535 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.816412 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.816790 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.816791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.816791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.816855 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.817047 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.817048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.817048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.817145 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.817275 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.817339 (+ 64us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.817413 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:46:46.819430 26712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.819564 26712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 085dff5bac7749b78f8b0a0566e864e9, State: Initialized, Role: FOLLOWER
I20260812 14:46:46.819705 26712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.819985 26848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.820055 26712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:46.820065 26848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:46.821578 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.814808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:45547 (local address 127.0.0.1:51120)
0812 14:46:46.814987 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.814997 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.815049 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.815213 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.815214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.815345 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.815397 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.815399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.816228 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.816229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.816821 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.816824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.817281 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.817282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.817349 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.817773 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.817777 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.818962 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.820592 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.820593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.820593 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.820795 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.820972 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.820973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.820973 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.821029 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.821325 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.821325 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.821363 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.821421 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.821502 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260812 14:46:46.821866 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.814852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.15.227.253:45547)
0812 14:46:46.814892 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.814893 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.815052 (+ 159us) server_negotiation.cc:407] Connection header received
0812 14:46:46.815098 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.815098 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.815108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.815162 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.815492 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.816116 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.816902 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.817159 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.817224 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.819046 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.819047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.819048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.819054 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.820516 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.820865 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.820866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.820867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.820921 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.821078 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.821079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.821079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.821234 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.821296 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.821711 (+ 415us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.821789 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:46:46.822212 26835 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } has no permanent_uuid. Determining permanent_uuid...
W20260812 14:46:46.822609 26873 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:46:46.822669 26873 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:46.822167 26591 master_runner.cc:186] Error getting master registration for 127.15.227.254:41209: OK, instance_id { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" instance_seqno: 1786546006534127 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:46:46.824690 26591 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:46:46.826287 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.818088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.15.227.254:41209)
0812 14:46:46.818129 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.818131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.818139 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:46.818187 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.818188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.818197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.818242 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.818816 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.819413 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.821449 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.821728 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.821767 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.823849 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.823850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.823851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.823858 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.825217 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.825570 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.825571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.825571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.825628 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.825807 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.825809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.825809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.825917 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.826070 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.826127 (+ 57us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.826207 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:46:46.831810 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.822405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55322)
0812 14:46:46.822463 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.822471 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.822500 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.824899 (+ 2399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.824900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.825013 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.825035 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.825036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.825751 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.825752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.826470 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.826473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.826951 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.826952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.826996 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.828102 (+ 1106us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.828107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.829292 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.830948 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.830949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.830949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.831097 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.831248 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.831248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.831248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.831303 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.831536 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.831537 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.831573 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.831633 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.831723 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 14:46:46.831867 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.824770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55322 (local address 127.15.227.252:35471)
0812 14:46:46.824810 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.824811 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.824818 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:46.824833 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.824833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.824841 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.824888 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.825091 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.825675 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.826563 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.826850 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.826910 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.829366 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.829367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.829367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.829373 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.830883 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.831153 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.831153 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.831154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.831198 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.831356 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.831357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.831358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.831480 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.831534 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.831657 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.831773 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":19}
I20260812 14:46:46.832963 26835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: Bootstrap starting.
I20260812 14:46:46.833545 26835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:46.833829 26835 log.cc:824] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:46.834579 26835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: No bootstrap required, opened a new log
I20260812 14:46:46.834594 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.826771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51130 (local address 127.15.227.253:45547)
0812 14:46:46.826800 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.826802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.826808 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:46:46.826825 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.826826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.826833 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.826876 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.827325 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.827987 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.829046 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.829221 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.829275 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.831760 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.831761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.831761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.831768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.833288 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.833724 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.833725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.833726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.833782 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.834060 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.834061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.834061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.834190 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.834375 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.834445 (+ 70us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.834522 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260812 14:46:46.835938 26835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.836045 26835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ef9fc6efb5544eabb6ebafece3279d2, State: Initialized, Role: FOLLOWER
I20260812 14:46:46.836115 26835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.836367 26835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:46.836367 26878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.836517 26878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:46.837692 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260812 14:46:46.838496 26897 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:46:46.838554 26897 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:46:46.838620 26533 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "dea8ba3f45f14666a460263a0e3a22d9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36722
I20260812 14:46:46.839066 26726 master_runner.cc:186] Error getting master registration for 127.15.227.254:41209: OK, instance_id { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" instance_seqno: 1786546006534127 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:46:46.840574 26726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:46:46.876976 26575 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } attempt: 3
Aug 12 14:46:46 dist-test-slave-q9h9 krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546006, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:46:46.884383 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.877241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:35471 (local address 127.0.0.1:55332)
0812 14:46:46.877293 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:46.877305 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:46.877349 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:46.877557 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:46.877569 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:46.877725 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:46.877763 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.877765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.878663 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.878664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.879458 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:46.879460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.879782 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.879784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:46.879837 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.880280 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:46.880285 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:46.881698 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:46.883453 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.883454 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.883454 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.883617 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.883802 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:46.883803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:46.883803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:46.883860 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:46.884098 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:46.884099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:46.884142 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 14:46:46.884200 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:46:46.884294 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:46:46.884383 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.877416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55332 (local address 127.15.227.252:35471)
0812 14:46:46.877465 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.877466 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.877473 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:46.877493 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.877494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.877503 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.877550 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.877833 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.878585 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.879515 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.879721 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.879776 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.881796 (+ 2020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.881798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.881798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.881806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.883388 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.883679 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.883680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.883681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.883735 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.883919 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.883920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.883920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.884036 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.884087 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.884232 (+ 145us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.884296 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260812 14:46:46.885957 26575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9: Bootstrap starting.
I20260812 14:46:46.886688 26575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:46.886996 26575 log.cc:824] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:46.887794 26575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9: No bootstrap required, opened a new log
I20260812 14:46:46.889068 26575 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.889205 26575 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea8ba3f45f14666a460263a0e3a22d9, State: Initialized, Role: FOLLOWER
I20260812 14:46:46.889282 26575 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.889587 26575 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:46.889635 26533 raft_consensus.cc:482] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 14:46:46.889688 26533 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:46.890524 26533 raft_consensus.cc:504] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.890805 26533 leader_election.cc:288] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [CANDIDATE]: Term 1 election: Requested vote from peers 085dff5bac7749b78f8b0a0566e864e9 (127.15.227.253:45547), 5ef9fc6efb5544eabb6ebafece3279d2 (127.15.227.252:35471)
I20260812 14:46:46.891182 26668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dea8ba3f45f14666a460263a0e3a22d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "085dff5bac7749b78f8b0a0566e864e9"
I20260812 14:46:46.891321 26668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [term 0 FOLLOWER]: Advancing to term 1
W20260812 14:46:46.891556 26919 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:46:46.891597 26919 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:46:46.892198 26668 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dea8ba3f45f14666a460263a0e3a22d9 in term 1.
I20260812 14:46:46.892776 26482 leader_election.cc:302] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 085dff5bac7749b78f8b0a0566e864e9, dea8ba3f45f14666a460263a0e3a22d9; no voters:
I20260812 14:46:46.893394 26788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dea8ba3f45f14666a460263a0e3a22d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5ef9fc6efb5544eabb6ebafece3279d2"
I20260812 14:46:46.893522 26788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:46.894336 26788 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dea8ba3f45f14666a460263a0e3a22d9 in term 1.
I20260812 14:46:46.896701 26452 master_runner.cc:263] No leader master found from master dea8ba3f45f14666a460263a0e3a22d9
I20260812 14:46:46.896745 26452 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:46:46.889562 26901 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.896916 26901 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:46.897006 26901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:46.897091 26901 raft_consensus.cc:686] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [term 1 LEADER]: Becoming Leader. State: Replica: dea8ba3f45f14666a460263a0e3a22d9, State: Running, Role: LEADER
I20260812 14:46:46.897215 26901 consensus_queue.cc:237] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } }
I20260812 14:46:46.897742 26901 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea8ba3f45f14666a460263a0e3a22d9. Latest consensus state: current_term: 1 leader_uuid: "dea8ba3f45f14666a460263a0e3a22d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.897867 26901 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:46.899000 26929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:46.899607 26929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:46.909052 26668 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [term 1 FOLLOWER]: Refusing update from remote peer dea8ba3f45f14666a460263a0e3a22d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:46.909085 26788 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [term 1 FOLLOWER]: Refusing update from remote peer dea8ba3f45f14666a460263a0e3a22d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:46.909372 26901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:46.909510 26901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:46.910756 26848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea8ba3f45f14666a460263a0e3a22d9. Latest consensus state: current_term: 1 leader_uuid: "dea8ba3f45f14666a460263a0e3a22d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.910840 26848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:46.910785 26878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dea8ba3f45f14666a460263a0e3a22d9. Latest consensus state: current_term: 1 leader_uuid: "dea8ba3f45f14666a460263a0e3a22d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.910864 26878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:46:46.913225 26622 tablet.cc:2369] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 14:46:46.918246 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.898774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.15.227.253:45547)
0812 14:46:46.898818 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.898820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.911018 (+ 12198us) server_negotiation.cc:407] Connection header received
0812 14:46:46.911104 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.911105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.911114 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.911141 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.911777 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.912443 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.913409 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.913689 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.913733 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.915383 (+ 1650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.915384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.915385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.915391 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.916944 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.917340 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.917342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.917342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.917399 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.917576 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.917577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.917577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.917712 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.917798 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.918081 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.918154 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:46:46.919257 26901 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dea8ba3f45f14666a460263a0e3a22d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.919345 26901 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:46.919788 26878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dea8ba3f45f14666a460263a0e3a22d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.919878 26878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:46.920145 26939 mvcc.cc:205] Tried to move back new op lower bound from 7317692444289519616 to 7317692444252127232. Current Snapshot: MvccSnapshot[applied={T|T < 7317692444289519616}]
I20260812 14:46:46.920657 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.898247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36726 (local address 127.15.227.254:41209)
0812 14:46:46.898285 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.898287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.900422 (+ 2135us) server_negotiation.cc:407] Connection header received
0812 14:46:46.900516 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.900517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.900527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.900554 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.902201 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.902899 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.905588 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.905891 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.905928 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.911886 (+ 5958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.911888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.911888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.911895 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.913418 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.914134 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.914135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.914136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.914192 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.919362 (+ 5170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.919363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.919363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.919502 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.919587 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.920491 (+ 904us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.920571 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260812 14:46:46.922098 26845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:46.898458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.15.227.252:35471)
0812 14:46:46.898492 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:46.898494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:46.901358 (+ 2864us) server_negotiation.cc:407] Connection header received
0812 14:46:46.901444 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:46.901445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:46.901454 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:46.901481 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:46.903920 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.904608 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.906903 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:46.907193 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:46.907232 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:46.913861 (+ 6629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:46.913862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:46.913863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:46.913870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:46.915487 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.920083 (+ 4596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.920084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.920084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.920144 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:46.921493 (+ 1349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:46.921494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:46.921494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:46.921629 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:46.921714 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:46.921915 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 14:46:46.921990 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:46.923698 26921 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dea8ba3f45f14666a460263a0e3a22d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.923787 26921 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:46.924192 26848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dea8ba3f45f14666a460263a0e3a22d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea8ba3f45f14666a460263a0e3a22d9" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 41209 } } peers { permanent_uuid: "085dff5bac7749b78f8b0a0566e864e9" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 45547 } } peers { permanent_uuid: "5ef9fc6efb5544eabb6ebafece3279d2" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 35471 } } }
I20260812 14:46:46.924269 26848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:46.925716 26941 mvcc.cc:205] Tried to move back new op lower bound from 7317692444289519616 to 7317692444252127232. Current Snapshot: MvccSnapshot[applied={T|T < 7317692444289519616}]
I20260812 14:46:46.930765 26929 catalog_manager.cc:1347] Generated new cluster ID: 5b52bf362d674c4da545cb756fc0c741
I20260812 14:46:46.930824 26929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:46.976452 26929 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:46.977173 26929 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:46.987071 26929 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dea8ba3f45f14666a460263a0e3a22d9: Generated new TSK 0
I20260812 14:46:46.987239 26929 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 14:46:47.008232 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.007310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51148 (local address 127.15.227.253:45547)
0812 14:46:47.007348 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.007350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.007775 (+ 425us) server_negotiation.cc:407] Connection header received
0812 14:46:47.007876 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.007877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.007888 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.007916 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.008174 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51148: BlockingRecv error: recv got EOF from 127.0.0.1:51148 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:46:47.008232 26582 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.007099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.15.227.254:41209)
0812 14:46:47.007152 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.007154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.007614 (+ 460us) server_negotiation.cc:407] Connection header received
0812 14:46:47.007673 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.007674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.007686 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.007736 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.008165 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36738: BlockingRecv error: recv got EOF from 127.0.0.1:36738 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260812 14:46:47.008955 26845 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.007471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55356 (local address 127.15.227.252:35471)
0812 14:46:47.007511 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.007512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.008481 (+ 969us) server_negotiation.cc:407] Connection header received
0812 14:46:47.008524 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.008525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.008535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.008579 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.008894 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55356: BlockingRecv error: recv got EOF from 127.0.0.1:55356 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260812 14:46:47.010557 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.009757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51150 (local address 127.15.227.253:45547)
0812 14:46:47.009795 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.009796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.010158 (+ 362us) server_negotiation.cc:407] Connection header received
0812 14:46:47.010224 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.010225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.010241 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.010285 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.010515 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51150: BlockingRecv error: recv got EOF from 127.0.0.1:51150 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260812 14:46:47.012941 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.012553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.15.227.253:45547)
0812 14:46:47.012584 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.012586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.012592 (+ 6us) server_negotiation.cc:407] Connection header received
0812 14:46:47.012611 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.012612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.012630 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.012672 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.012902 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51158: BlockingRecv error: recv got EOF from 127.0.0.1:51158 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:46:47.016144 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.015646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51166 (local address 127.15.227.253:45547)
0812 14:46:47.015682 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.015683 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.015780 (+ 97us) server_negotiation.cc:407] Connection header received
0812 14:46:47.015829 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.015830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.015840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.015882 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.016105 (+ 223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51166: BlockingRecv error: recv got EOF from 127.0.0.1:51166 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:46:47.021517 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.021022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51182 (local address 127.15.227.253:45547)
0812 14:46:47.021066 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.021068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.021159 (+ 91us) server_negotiation.cc:407] Connection header received
0812 14:46:47.021210 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.021210 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.021219 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.021262 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.021481 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51182: BlockingRecv error: recv got EOF from 127.0.0.1:51182 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260812 14:46:47.030944 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.030415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.15.227.253:45547)
0812 14:46:47.030447 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.030449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.030559 (+ 110us) server_negotiation.cc:407] Connection header received
0812 14:46:47.030608 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.030608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.030626 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.030669 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.030902 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51188: BlockingRecv error: recv got EOF from 127.0.0.1:51188 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:46:47.048878 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.048013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51190 (local address 127.15.227.253:45547)
0812 14:46:47.048052 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.048054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.048296 (+ 242us) server_negotiation.cc:407] Connection header received
0812 14:46:47.048410 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.048412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.048424 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.048459 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.048824 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51190: BlockingRecv error: recv got EOF from 127.0.0.1:51190 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:46:47.082893 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.082197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.15.227.253:45547)
0812 14:46:47.082248 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.082250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.082379 (+ 129us) server_negotiation.cc:407] Connection header received
0812 14:46:47.082445 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.082446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.082456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.082499 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.082829 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51204: BlockingRecv error: recv got EOF from 127.0.0.1:51204 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260812 14:46:47.148558 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.147855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.15.227.253:45547)
0812 14:46:47.147922 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.147925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.148030 (+ 105us) server_negotiation.cc:407] Connection header received
0812 14:46:47.148081 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.148082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.148096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.148156 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.148501 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51214: BlockingRecv error: recv got EOF from 127.0.0.1:51214 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260812 14:46:47.278258 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.277542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.15.227.253:45547)
0812 14:46:47.277619 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.277621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.277686 (+ 65us) server_negotiation.cc:407] Connection header received
0812 14:46:47.277760 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.277760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.277771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.277812 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.278185 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51226: BlockingRecv error: recv got EOF from 127.0.0.1:51226 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260812 14:46:47.535882 26717 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:47.535252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.15.227.253:45547)
0812 14:46:47.535302 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:47.535304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:47.535387 (+ 83us) server_negotiation.cc:407] Connection header received
0812 14:46:47.535456 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:47.535457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:47.535468 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:47.535516 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:47.535826 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51230: BlockingRecv error: recv got EOF from 127.0.0.1:51230 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260812 14:46:47.823061 26873 catalog_manager.cc:1259] Loaded cluster ID: 5b52bf362d674c4da545cb756fc0c741
I20260812 14:46:47.823127 26873 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: loading cluster ID for follower catalog manager: success
I20260812 14:46:47.823899 26873 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: acquiring CA information for follower catalog manager: success
I20260812 14:46:47.824196 26873 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 085dff5bac7749b78f8b0a0566e864e9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:46:47.838891 26897 catalog_manager.cc:1259] Loaded cluster ID: 5b52bf362d674c4da545cb756fc0c741
I20260812 14:46:47.838943 26897 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: loading cluster ID for follower catalog manager: success
I20260812 14:46:47.839673 26897 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: acquiring CA information for follower catalog manager: success
I20260812 14:46:47.839949 26897 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ef9fc6efb5544eabb6ebafece3279d2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:46:48.051244 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:48.048905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47756 (local address 127.15.227.253:45547)
0812 14:46:48.049007 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:48.049009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:48.049064 (+ 55us) server_negotiation.cc:407] Connection header received
0812 14:46:48.049130 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:48.049130 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:48.049184 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:48.049261 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:48.049377 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.049866 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.050560 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.050768 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.050815 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:48.050964 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:48.051077 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:48.051130 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:48.051159 (+ 29us) server_negotiation.cc:299] Negotiation successful
0812 14:46:48.051177 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":54,"threads_started":1}
I20260812 14:46:48.055495 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:48.052873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41180 (local address 127.15.227.252:35471)
0812 14:46:48.052963 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:48.052965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:48.052972 (+ 7us) server_negotiation.cc:407] Connection header received
0812 14:46:48.053006 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:48.053006 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:48.053040 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:48.053106 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:48.053176 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.053664 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.054689 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.054920 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.054977 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:48.055140 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:48.055277 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:48.055341 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:48.055392 (+ 51us) server_negotiation.cc:299] Negotiation successful
0812 14:46:48.055412 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":46,"threads_started":1}
I20260812 14:46:48.059846 27084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:48.057264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.15.227.253:45547)
0812 14:46:48.057299 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:48.057301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:48.057420 (+ 119us) server_negotiation.cc:407] Connection header received
0812 14:46:48.057515 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:48.057516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:48.057526 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:48.057550 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:48.057723 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.058444 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.059273 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.059469 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.059530 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:48.059640 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:48.059667 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:48.059705 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:48.059760 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 14:46:48.059778 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:48.060230 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:48.057140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.15.227.254:41209)
0812 14:46:48.057357 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:48.057359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:48.057458 (+ 99us) server_negotiation.cc:407] Connection header received
0812 14:46:48.057528 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:48.057529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:48.057578 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:48.057644 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:48.057779 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.058318 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.059335 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.059644 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.059690 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:48.059898 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:48.059969 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:48.060023 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:48.060116 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 14:46:48.060142 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":62,"threads_started":1}
I20260812 14:46:48.060462 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:48.057197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.15.227.252:35471)
0812 14:46:48.057276 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:48.057278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:48.057479 (+ 201us) server_negotiation.cc:407] Connection header received
0812 14:46:48.057502 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:48.057503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:48.057513 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:48.057558 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:48.057689 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.058429 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.059496 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:48.059768 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:48.059823 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:48.060095 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:48.060154 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:48.060202 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:48.060261 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 14:46:48.060280 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 14:46:48.061805 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26452
I20260812 14:46:48.073701 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26591
I20260812 14:46:48.084704 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 26726
2026-08-12T14:46:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2959 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 14:46:48 dist-test-slave-q9h9 krb5kdc[27114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 14:46:48 dist-test-slave-q9h9 krb5kdc[27114](info): set up 2 sockets
Aug 12 14:46:48 dist-test-slave-q9h9 krb5kdc[27114](info): commencing operation
krb5kdc: starting...
W20260812 14:46:49.280550 16271 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/test-user.keytab.
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T14:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T14:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.227.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
Entry for principal kudu/127.15.227.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab.
I20260812 14:46:49.388833 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:40191
--webserver_interface=127.15.227.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:40191,127.15.227.253:38593,127.15.227.252:34665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 14:46:49.466758 27142 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:46:49.466949 27142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:49.466972 27142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:49.466998 27142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:49.468111 27142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:49.468150 27142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:49.468178 27142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:49.468195 27142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:49.468216 27142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:49.469652 27142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:40191,127.15.227.253:38593,127.15.227.252:34665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.254.keytab
--principal=kudu/127.15.227.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.254:40191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.227.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:49.469895 27142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:49.470113 27142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:46:49.474541 27142 server_base.cc:1034] running on GCE node
W20260812 14:46:49.474498 27157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:49.474512 27151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:49.474500 27153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:49.475021 27142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:49.475256 27142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:49.476398 27142 hybrid_clock.cc:648] HybridClock initialized: now 1786546009476385 us; error 26 us; skew 500 ppm
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:49.478657 27142 init.cc:377] Logged in from keytab as kudu/127.15.227.254@KRBTEST.COM (short username kudu)
I20260812 14:46:49.479014 27142 webserver.cc:460] Webserver started at http://127.15.227.254:36179/ using document root <none> and password file <none>
I20260812 14:46:49.479183 27142 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:49.479223 27142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:49.479310 27142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:49.480207 27142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data/instance:
uuid: "0a7f46c254e64b2886924ddca7096198"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "d018b8a1771886bc69d1673207562c36"
server_key_iv: "609c6d1b588a81f87462eb5222ae10f2"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.480479 27142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/wal/instance:
uuid: "0a7f46c254e64b2886924ddca7096198"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "d018b8a1771886bc69d1673207562c36"
server_key_iv: "609c6d1b588a81f87462eb5222ae10f2"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.481591 27142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:49.482244 27168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:49.482380 27142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:49.482462 27142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/wal
uuid: "0a7f46c254e64b2886924ddca7096198"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "d018b8a1771886bc69d1673207562c36"
server_key_iv: "609c6d1b588a81f87462eb5222ae10f2"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.482522 27142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:49.493458 27142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:49.495050 27142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:49.495173 27142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:49.498832 27142 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.254:40191
I20260812 14:46:49.498880 27254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.254:40191 every 8 connection(s)
I20260812 14:46:49.499164 27142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/data/info.pb
I20260812 14:46:49.499929 27256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:49.501266 27256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:49.502408 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 27142
I20260812 14:46:49.502493 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-0/wal/instance
I20260812 14:46:49.502663 16271 external_mini_cluster.cc:1428] Setting key fa32928b5d32ac9643fb4d182d7c061c
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:49.510509 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.501816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40191 (local address 127.0.0.1:50750)
0812 14:46:49.501954 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.501987 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.502471 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.502747 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.502750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.502981 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.503118 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.503122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.504121 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.504122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.504825 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.504828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.505359 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.505360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.505412 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.505825 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.505832 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.507254 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.509154 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.509161 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.509162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.509353 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.509606 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.509606 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.509606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.509682 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.510084 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.510087 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.510140 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.510220 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.510335 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260812 14:46:49.510526 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.501955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50750 (local address 127.15.227.254:40191)
0812 14:46:49.502079 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.502082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.502107 (+ 25us) server_negotiation.cc:407] Connection header received
0812 14:46:49.502637 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.502639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.502654 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.502722 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.503213 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.504009 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.504913 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.505296 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.505376 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.507342 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.507352 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.507352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.507360 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.509078 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.509431 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.509434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.509434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.509537 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.509742 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.509742 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.509743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.510004 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.510111 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.510273 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.510409 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":99,"thread_start_us":36,"threads_started":1}
WARNING: no policy specified for kudu/127.15.227.253@KRBTEST.COM; defaulting to no policy
I20260812 14:46:49.511194 27256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:49.511502 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.511385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60616)
0812 14:46:49.511431 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.511478 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260812 14:46:49.511617 27169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
I20260812 14:46:49.511957 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.511884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60620)
0812 14:46:49.511919 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.511943 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 14:46:49.512060 27256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:38593: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab.
Entry for principal kudu/127.15.227.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab.
I20260812 14:46:49.526434 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:38593
--webserver_interface=127.15.227.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab
--principal=kudu/127.15.227.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:40191,127.15.227.253:38593,127.15.227.252:34665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:46:49.561245 27256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } attempt: 1
I20260812 14:46:49.561693 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.561575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60632)
0812 14:46:49.561630 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.561666 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260812 14:46:49.562041 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.561936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60638)
0812 14:46:49.561975 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.561999 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 14:46:49.562119 27256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:38593: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
W20260812 14:46:49.604996 27270 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:46:49.605182 27270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:49.605208 27270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:49.605232 27270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:49.606360 27270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:49.606396 27270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:49.606416 27270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:49.606431 27270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:49.606446 27270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:49.607884 27270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:40191,127.15.227.253:38593,127.15.227.252:34665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.253.keytab
--principal=kudu/127.15.227.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.253:38593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.227.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:49.608116 27270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:49.608304 27270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 14:46:49.612680 27282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:49.612795 27288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:49.612751 27290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:49.612753 27270 server_base.cc:1034] running on GCE node
I20260812 14:46:49.613087 27270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:49.613355 27270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:49.614488 27270 hybrid_clock.cc:648] HybridClock initialized: now 1786546009614475 us; error 31 us; skew 500 ppm
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:49.616712 27270 init.cc:377] Logged in from keytab as kudu/127.15.227.253@KRBTEST.COM (short username kudu)
I20260812 14:46:49.617082 27270 webserver.cc:460] Webserver started at http://127.15.227.253:32899/ using document root <none> and password file <none>
I20260812 14:46:49.617280 27270 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:49.617326 27270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:49.617447 27270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:49.618428 27270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data/instance:
uuid: "a8b044417baf49fa8767ce38ba0e68f1"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "c033f54d5af8d771317a7d1f3e0e4c24"
server_key_iv: "1c6aa4c629b98f12c7f2e92dfdcd22ba"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.618743 27270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/wal/instance:
uuid: "a8b044417baf49fa8767ce38ba0e68f1"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "c033f54d5af8d771317a7d1f3e0e4c24"
server_key_iv: "1c6aa4c629b98f12c7f2e92dfdcd22ba"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.619902 27270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260812 14:46:49.620550 27303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:49.620720 27270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:49.620803 27270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/wal
uuid: "a8b044417baf49fa8767ce38ba0e68f1"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "c033f54d5af8d771317a7d1f3e0e4c24"
server_key_iv: "1c6aa4c629b98f12c7f2e92dfdcd22ba"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.620862 27270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:49.630221 27256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } attempt: 2
I20260812 14:46:49.630671 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.630548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60640)
0812 14:46:49.630602 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.630638 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:46:49.631006 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.630928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60656)
0812 14:46:49.630965 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.630984 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 14:46:49.631579 27256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.253:38593: Network error: Client connection negotiation failed: client connection to 127.15.227.253:38593: connect: Connection refused (error 111)
I20260812 14:46:49.637518 27270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:49.639104 27270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:49.639204 27270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:49.642896 27270 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.253:38593
I20260812 14:46:49.642962 27396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.253:38593 every 8 connection(s)
I20260812 14:46:49.643214 27270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/data/info.pb
I20260812 14:46:49.643985 27397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:49.645318 27397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:49.650475 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 27270
I20260812 14:46:49.650578 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-1/wal/instance
I20260812 14:46:49.650782 16271 external_mini_cluster.cc:1428] Setting key ea19df6770d2fd5b1b5057351424660e
I20260812 14:46:49.653008 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.645881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.15.227.254:40191)
0812 14:46:49.645933 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.645935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.646137 (+ 202us) server_negotiation.cc:407] Connection header received
0812 14:46:49.646559 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.646560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.646568 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.646618 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.647109 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.647563 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.648395 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.648573 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.648614 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.650388 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.650390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.650390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.650398 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.651764 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.652107 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.652109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.652109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.652170 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.652362 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.652362 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.652362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.652488 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.652550 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.652874 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.652950 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 14:46:49.653158 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.645850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40191 (local address 127.0.0.1:50760)
0812 14:46:49.646028 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.646067 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.646492 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.646663 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.646666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.646872 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.647046 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.647050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.647616 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.647616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.648339 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.648341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.648623 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.648623 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.648714 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.649067 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.649074 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.650302 (+ 1228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.651836 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.651843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.651845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.652043 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.652224 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.652224 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.652224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.652309 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.652549 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.652554 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.652710 (+ 156us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.652818 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.652964 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260812 14:46:49.653792 27397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.227.252@KRBTEST.COM; defaulting to no policy
I20260812 14:46:49.661422 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.654033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60666)
0812 14:46:49.654104 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.654115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.654157 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.654359 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.654360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.654492 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.654546 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.654547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.655691 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.655692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.656384 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.656387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.656991 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.656991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.657045 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.657389 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.657392 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.658683 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.660484 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.660485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.660485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.660684 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.660867 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.660867 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.660867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.660907 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.661180 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.661181 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.661212 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.661261 (+ 49us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.661337 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 14:46:49.661732 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.654104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.15.227.253:38593)
0812 14:46:49.654233 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.654237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.654255 (+ 18us) server_negotiation.cc:407] Connection header received
0812 14:46:49.654275 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.654276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.654287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.654344 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.654619 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.655596 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.656463 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.656886 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.656939 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.658759 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.658767 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.658767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.658773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.660366 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.660769 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.660771 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.660771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.660818 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.660952 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.660952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.660953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.661099 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.661162 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.661550 (+ 388us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.661643 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.227.252@KRBTEST.COM" created.
I20260812 14:46:49.664211 27397 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:49.665853 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.664414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46574)
0812 14:46:49.665783 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.665832 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1350}
W20260812 14:46:49.665983 27307 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
I20260812 14:46:49.666355 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.666292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46580)
0812 14:46:49.666316 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.666339 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260812 14:46:49.666448 27397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:34665: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.227.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab.
Entry for principal kudu/127.15.227.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab.
I20260812 14:46:49.676257 16271 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskN3u55Q/build/release/bin/kudu
/tmp/dist-test-taskN3u55Q/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:34665
--webserver_interface=127.15.227.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab
--principal=kudu/127.15.227.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.227.212:34785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.227.254:40191,127.15.227.253:38593,127.15.227.252:34665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 14:46:49.715641 27397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } attempt: 1
I20260812 14:46:49.716009 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.715911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46586)
0812 14:46:49.715956 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.715986 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260812 14:46:49.716332 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.716264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46602)
0812 14:46:49.716297 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.716317 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260812 14:46:49.716408 27397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:34665: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
I20260812 14:46:49.722728 27256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } attempt: 3
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:46:49.729522 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.723012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60678)
0812 14:46:49.723070 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.723081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.723130 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.723350 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.723352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.723537 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.723609 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.723611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.724292 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.724293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.725143 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.725146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.725487 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.725488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.725569 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.725931 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.725936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.727188 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.728712 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.728713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.728714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.728867 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.729048 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.729049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.729049 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.729092 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.729311 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.729311 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.729349 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.729396 (+ 47us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.729463 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260812 14:46:49.729566 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.723197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.15.227.253:38593)
0812 14:46:49.723241 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.723243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.723255 (+ 12us) server_negotiation.cc:407] Connection header received
0812 14:46:49.723278 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.723279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.723287 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.723332 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.723680 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.724203 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.725214 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.725423 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.725471 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.727263 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.727264 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.727265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.727272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.728637 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.728928 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.728929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.728930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.728989 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.729137 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.729138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.729138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.729260 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.729320 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.729419 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.729486 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260812 14:46:49.730046 27256 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:49.730334 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.730248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46608)
0812 14:46:49.730287 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.730314 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260812 14:46:49.730623 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.730546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46622)
0812 14:46:49.730583 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.730605 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260812 14:46:49.730726 27256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:34665: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
W20260812 14:46:49.757014 27416 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 14:46:49.757189 27416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 14:46:49.757212 27416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 14:46:49.757236 27416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 14:46:49.758330 27416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 14:46:49.758368 27416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 14:46:49.758380 27416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 14:46:49.758391 27416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 14:46:49.758404 27416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 14:46:49.759791 27416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.227.212:34785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.227.254:40191,127.15.227.253:38593,127.15.227.252:34665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/krb5kdc/kudu_127.15.227.252.keytab
--principal=kudu/127.15.227.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.227.252:34665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.227.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 12 Aug 2026 13:51:05 UTC on ccycloud.quasar-hjnkdx.root.comops.site
build id 4582765
I20260812 14:46:49.759954 27416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 14:46:49.760113 27416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 14:46:49.761860 27256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } attempt: 1
I20260812 14:46:49.762141 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.762049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46638)
0812 14:46:49.762085 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.762117 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260812 14:46:49.762442 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.762364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46644)
0812 14:46:49.762402 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.762425 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260812 14:46:49.762531 27256 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:34665: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
W20260812 14:46:49.764544 27429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:49.764582 27423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 14:46:49.764555 27426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 14:46:49.764846 27416 server_base.cc:1034] running on GCE node
I20260812 14:46:49.765041 27416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 14:46:49.765280 27416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 14:46:49.766409 27416 hybrid_clock.cc:648] HybridClock initialized: now 1786546009766396 us; error 27 us; skew 500 ppm
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 14:46:49.768504 27416 init.cc:377] Logged in from keytab as kudu/127.15.227.252@KRBTEST.COM (short username kudu)
I20260812 14:46:49.768901 27416 webserver.cc:460] Webserver started at http://127.15.227.252:42259/ using document root <none> and password file <none>
I20260812 14:46:49.769101 27416 fs_manager.cc:359] Metadata directory not provided
I20260812 14:46:49.769145 27416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 14:46:49.769268 27416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 14:46:49.770254 27416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data/instance:
uuid: "57159baf2a2c4c64b0a6bfd201ade9d5"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "ad3c9ec718a5c6e619f3c3df7f355ef5"
server_key_iv: "d51e4b64231145a187fab2a2a3f2b0a8"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.770556 27416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/wal/instance:
uuid: "57159baf2a2c4c64b0a6bfd201ade9d5"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "ad3c9ec718a5c6e619f3c3df7f355ef5"
server_key_iv: "d51e4b64231145a187fab2a2a3f2b0a8"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.771745 27416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260812 14:46:49.772398 27443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 14:46:49.772552 27416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260812 14:46:49.772630 27416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data,/tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/wal
uuid: "57159baf2a2c4c64b0a6bfd201ade9d5"
format_stamp: "Formatted at 2026-08-12 14:46:49 on dist-test-slave-q9h9"
server_key: "ad3c9ec718a5c6e619f3c3df7f355ef5"
server_key_iv: "d51e4b64231145a187fab2a2a3f2b0a8"
server_key_version: "encryptionkey@0"
I20260812 14:46:49.772689 27416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 14:46:49.781661 27416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 14:46:49.783380 27416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 14:46:49.783499 27416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 14:46:49.784569 27397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } attempt: 2
I20260812 14:46:49.785014 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.784888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46656)
0812 14:46:49.784948 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.784981 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 14:46:49.785389 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.785294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46668)
0812 14:46:49.785335 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.785364 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260812 14:46:49.785499 27397 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.227.252:34665: Network error: Client connection negotiation failed: client connection to 127.15.227.252:34665: connect: Connection refused (error 111)
I20260812 14:46:49.787451 27416 rpc_server.cc:307] RPC server started. Bound to: 127.15.227.252:34665
I20260812 14:46:49.787478 27550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.227.252:34665 every 8 connection(s)
I20260812 14:46:49.787777 27416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/data/info.pb
I20260812 14:46:49.788546 27551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 14:46:49.789975 16271 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskN3u55Q/build/release/bin/kudu as pid 27416
I20260812 14:46:49.790086 16271 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskN3u55Q/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786545740023890-16271-0/minicluster-data/master-2/wal/instance
I20260812 14:46:49.789983 27551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:49.790285 16271 external_mini_cluster.cc:1428] Setting key 8716b4ed328feccc33d9e9f5551f74df
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:46:49.798730 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.790667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.15.227.254:40191)
0812 14:46:49.790748 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.790750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.790937 (+ 187us) server_negotiation.cc:407] Connection header received
0812 14:46:49.791420 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.791421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.791433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.791480 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.791986 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.792452 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.793278 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.793463 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.793517 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.795732 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.795734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.795734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.795740 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.797274 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.797739 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.797740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.797741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.797802 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.798094 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.798094 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.798095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.798220 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.798286 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.798569 (+ 283us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.798646 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 14:46:49.798882 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.790648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.254:40191 (local address 127.0.0.1:50762)
0812 14:46:49.790826 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.790859 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.791333 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.791518 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.791520 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.791758 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.791904 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.791908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.792517 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.792518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.793199 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.793201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.793528 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.793529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.793641 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.794130 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.794140 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.795658 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.797367 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.797374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.797376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.797659 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.797877 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.797878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.797878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.798008 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.798294 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.798301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.798450 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.798528 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.798664 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":83,"threads_started":1}
I20260812 14:46:49.798979 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.790663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.15.227.252:34665)
0812 14:46:49.790829 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.790840 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.790872 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:46:49.791341 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.791347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.791365 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.791436 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.792028 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.792806 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.793512 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.793819 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.793880 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.796211 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.796220 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.796220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.796227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.797773 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.798186 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.798189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.798190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.798246 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.798422 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.798422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.798422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.798555 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.798714 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.798770 (+ 56us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.798847 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260812 14:46:49.799499 27551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.254@KRBTEST.COM
I20260812 14:46:49.805371 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.799717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.253:38593 (local address 127.0.0.1:60688)
0812 14:46:49.799849 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.799859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.799900 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.800005 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.800006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.800136 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.800188 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.800189 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.800742 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.800743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.801244 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.801246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.801595 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.801596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.801659 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.801976 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.801980 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.803154 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.804496 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.804497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.804498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.804662 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.804825 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.804825 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.804826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.804879 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.805118 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.805119 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.805155 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.805215 (+ 60us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.805289 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 14:46:49.805390 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.799811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60688 (local address 127.15.227.253:38593)
0812 14:46:49.799848 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.799850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.799890 (+ 40us) server_negotiation.cc:407] Connection header received
0812 14:46:49.799954 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.799955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.799962 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.799999 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.800244 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.800673 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.801323 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.801539 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.801591 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.803238 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.803239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.803240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.803247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.804424 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.804721 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.804722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.804723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.804773 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.804927 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.804928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.804929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.805043 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.805108 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.805254 (+ 146us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.805323 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:49.805923 27551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } has no permanent_uuid. Determining permanent_uuid...
I20260812 14:46:49.806005 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.799663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50770 (local address 127.15.227.254:40191)
0812 14:46:49.799701 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.799703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.799711 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:49.799752 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.799753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.799762 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.799808 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.800335 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.800765 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.801377 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.801565 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.801615 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.803742 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.803743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.803744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.803749 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.804971 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.805291 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.805292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.805292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.805346 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.805522 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.805524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.805524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.805644 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.805803 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.805866 (+ 63us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.805944 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.252@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.227.253@KRBTEST.COM
I20260812 14:46:49.811782 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.806176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46698)
0812 14:46:49.806221 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.806232 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.806274 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.806394 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.806395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.806525 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.806552 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.806554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.807243 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.807244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.807723 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.807725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.808043 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.808044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.808100 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.808404 (+ 304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.808408 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.809458 (+ 1050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.810896 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.810897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.810897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.811073 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.811255 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.811255 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.811256 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.811308 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.811501 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.811502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.811537 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.811595 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.811692 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260812 14:46:49.811839 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.806200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.15.227.252:34665)
0812 14:46:49.806232 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.806233 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.806260 (+ 27us) server_negotiation.cc:407] Connection header received
0812 14:46:49.806313 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.806314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.806322 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.806363 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.806622 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.807175 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.807795 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.807983 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.808026 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.809527 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.809528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.809528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.809534 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.810829 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.811147 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.811148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.811149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.811204 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.811357 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.811357 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.811358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.811451 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.811502 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.811621 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.811741 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":20}
I20260812 14:46:49.813376 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.806771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60700 (local address 127.15.227.253:38593)
0812 14:46:49.806803 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.806805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.806834 (+ 29us) server_negotiation.cc:407] Connection header received
0812 14:46:49.806850 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.806851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.806859 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.806908 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.807302 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.807707 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.808433 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.808606 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.808659 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.811233 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.811235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.811235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.811242 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.812486 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.812804 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.812804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.812805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.812846 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.812992 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.812992 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.812993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.813069 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.813185 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.813265 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.813321 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:49.813446 27551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: Bootstrap starting.
I20260812 14:46:49.814476 27551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:49.814832 27551 log.cc:824] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:49.815088 16271 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 14:46:49.815618 27551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: No bootstrap required, opened a new log
I20260812 14:46:49.816130 27212 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0a7f46c254e64b2886924ddca7096198"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50770
I20260812 14:46:49.817047 27551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.817173 27551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57159baf2a2c4c64b0a6bfd201ade9d5, State: Initialized, Role: FOLLOWER
I20260812 14:46:49.817234 27551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.817466 27551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:49.817472 27561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.817579 27561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:46:49.819089 27580 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:46:49.819141 27580 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:46:49.819559 27416 master_runner.cc:186] Error getting master registration for 127.15.227.254:40191: OK, instance_id { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" instance_seqno: 1786546009497789 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:46:49.822189 27416 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:46:49.837677 27256 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } attempt: 2
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.254@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:46:49.844558 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.837980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.15.227.252:34665)
0812 14:46:49.838044 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.838045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.838053 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:49.838071 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.838072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.838080 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.838126 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.838412 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.839138 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.840102 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.840396 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.840456 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.842266 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.842268 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.842268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.842275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.843486 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.843852 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.843853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.843853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.843906 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.844073 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.844073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.844073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.844217 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.844278 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.844405 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.844474 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 14:46:49.844588 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.837915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46700)
0812 14:46:49.837966 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.837976 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.838026 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.838129 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.838130 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.838293 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.838336 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.838337 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.839212 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.839213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.840025 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.840027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.840467 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.840468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.840520 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.840876 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.840881 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.842188 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.843556 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.843556 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.843557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.843780 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.843961 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.843961 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.843962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.844021 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.844276 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.844277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.844321 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.844386 (+ 65us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.844495 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260812 14:46:49.846221 27256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198: Bootstrap starting.
I20260812 14:46:49.846925 27256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:49.847234 27256 log.cc:824] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:49.848001 27256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198: No bootstrap required, opened a new log
I20260812 14:46:49.849325 27256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.849463 27256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7f46c254e64b2886924ddca7096198, State: Initialized, Role: FOLLOWER
I20260812 14:46:49.849543 27256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.849843 27256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:49.849893 27212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 14:46:49.849956 27212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:49.850687 27212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.851135 27503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a7f46c254e64b2886924ddca7096198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5"
I20260812 14:46:49.851250 27503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:49.849812 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.851574 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:49.852090 27503 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a7f46c254e64b2886924ddca7096198 in term 1.
W20260812 14:46:49.852332 27595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260812 14:46:49.852602 27170 leader_election.cc:302] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a7f46c254e64b2886924ddca7096198, 57159baf2a2c4c64b0a6bfd201ade9d5; no voters:
I20260812 14:46:49.852737 27584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 14:46:49.852861 27584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [term 1 LEADER]: Becoming Leader. State: Replica: 0a7f46c254e64b2886924ddca7096198, State: Running, Role: LEADER
I20260812 14:46:49.852938 27584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.853338 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7f46c254e64b2886924ddca7096198. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.853399 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:49.853507 27212 leader_election.cc:288] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [CANDIDATE]: Term 1 election: Requested vote from peers a8b044417baf49fa8767ce38ba0e68f1 (127.15.227.253:38593), 57159baf2a2c4c64b0a6bfd201ade9d5 (127.15.227.252:34665)
W20260812 14:46:49.852399 27595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:46:49.852401 27142 master_runner.cc:186] Error getting master registration for 127.15.227.253:38593: OK, instance_id { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" instance_seqno: 1786546009641834 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 14:46:49.854537 27142 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 14:46:49.859056 27603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 14:46:49.859277 27603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 14:46:49.860986 27503 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [term 1 FOLLOWER]: Refusing update from remote peer 0a7f46c254e64b2886924ddca7096198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:49.861280 27584 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:49.862576 27561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7f46c254e64b2886924ddca7096198. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.862659 27561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:49.867177 27598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.867260 27598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: This master's current role is: LEADER
I20260812 14:46:49.867506 27615 mvcc.cc:205] Tried to move back new op lower bound from 7317692456387309568 to 7317692456358395904. Current Snapshot: MvccSnapshot[applied={T|T < 7317692456387309568}]
I20260812 14:46:49.867686 27561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.867769 27561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:49.867827 27616 mvcc.cc:205] Tried to move back new op lower bound from 7317692456387309568 to 7317692456358395904. Current Snapshot: MvccSnapshot[applied={T|T < 7317692456387309568}]
I20260812 14:46:49.869961 27262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.858285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.15.227.254:40191)
0812 14:46:49.858331 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.858333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.858828 (+ 495us) server_negotiation.cc:407] Connection header received
0812 14:46:49.858974 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.858975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.858999 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.859029 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.860286 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.860916 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.862779 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.862996 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.863036 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.864585 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.864586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.864587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.864593 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.866367 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.866980 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.866981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.866982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.867054 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.868100 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.868101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.868102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.868235 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.868326 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.869543 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.869875 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260812 14:46:49.872574 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.858473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60702 (local address 127.15.227.253:38593)
0812 14:46:49.858529 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.858531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.863506 (+ 4975us) server_negotiation.cc:407] Connection header received
0812 14:46:49.863622 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.863623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.863634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.863664 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.865094 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.865763 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.866859 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.867139 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.867186 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.869376 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.869377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.869378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.869384 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.870909 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.871522 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.871524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.871524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.871599 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.871911 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.871912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.871913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.872033 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.872095 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.872413 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.872488 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260812 14:46:49.875187 27603 catalog_manager.cc:1347] Generated new cluster ID: 3a25b5015a9a421ea1691c89a2cc4460
I20260812 14:46:49.875238 27603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 14:46:49.876638 27397 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } attempt: 3
I20260812 14:46:49.883489 27603 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 14:46:49.884140 27603 catalog_manager.cc:1504] Loading token signing keys...
I20260812 14:46:49.888839 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.858519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46706 (local address 127.15.227.252:34665)
0812 14:46:49.858551 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.858552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.866763 (+ 8211us) server_negotiation.cc:407] Connection header received
0812 14:46:49.866825 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.866826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.866836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.866885 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.871641 (+ 4756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.872271 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.873155 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.873404 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.873442 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.874969 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.874971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.874971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.874978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.876678 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.882756 (+ 6078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.882757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.882758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.882818 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.883022 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.883023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.883024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.883144 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.883213 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.888681 (+ 5468us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.888762 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260812 14:46:49.891251 27603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198: Generated new TSK 0
I20260812 14:46:49.891422 27603 catalog_manager.cc:1514] Initializing in-progress tserver states...
Aug 12 14:46:49 dist-test-slave-q9h9 krb5kdc[27114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786546009, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.227.253@KRBTEST.COM for kudu/127.15.227.252@KRBTEST.COM
I20260812 14:46:49.895921 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.882166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.227.252:34665 (local address 127.0.0.1:46716)
0812 14:46:49.882209 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0812 14:46:49.882218 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0812 14:46:49.882252 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 14:46:49.889018 (+ 6766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 14:46:49.889019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 14:46:49.889168 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0812 14:46:49.889205 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.889207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.889967 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.889968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.890780 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 14:46:49.890782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.891586 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.891587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 14:46:49.891643 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.891993 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 14:46:49.891998 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0812 14:46:49.893251 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 14:46:49.894900 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.894901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.894902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.895095 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.895320 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 14:46:49.895321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 14:46:49.895321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0812 14:46:49.895377 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 14:46:49.895642 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 14:46:49.895643 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 14:46:49.895685 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 14:46:49.895743 (+ 58us) client_negotiation.cc:240] Negotiation successful
0812 14:46:49.895831 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260812 14:46:49.896230 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:49.882212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.15.227.252:34665)
0812 14:46:49.888901 (+ 6689us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.888902 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.888910 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:49.888925 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.888926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.888934 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.888979 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.889283 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.889887 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.890855 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:49.891111 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:49.891185 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:49.893332 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 14:46:49.893333 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 14:46:49.893334 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 14:46:49.893340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0812 14:46:49.894816 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.895190 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.895191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.895191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.895261 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 14:46:49.895423 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 14:46:49.895424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 14:46:49.895424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0812 14:46:49.895554 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 14:46:49.895609 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:49.895813 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 14:46:49.895932 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6676,"thread_start_us":85,"threads_started":1}
I20260812 14:46:49.897668 27397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: Bootstrap starting.
I20260812 14:46:49.899112 27397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: Neither blocks nor log segments found. Creating new log.
I20260812 14:46:49.899435 27397 log.cc:824] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: Log is configured to *not* fsync() on all Append() calls
I20260812 14:46:49.900540 27397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: No bootstrap required, opened a new log
I20260812 14:46:49.902441 27397 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.902565 27397 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8b044417baf49fa8767ce38ba0e68f1, State: Initialized, Role: FOLLOWER
I20260812 14:46:49.902637 27397 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } }
I20260812 14:46:49.902927 27397 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 14:46:49.903169 27354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 14:46:49.904002 27354 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [term 1 FOLLOWER]: Refusing update from remote peer 0a7f46c254e64b2886924ddca7096198: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 14:46:49.904166 27356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0a7f46c254e64b2886924ddca7096198" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a8b044417baf49fa8767ce38ba0e68f1"
I20260812 14:46:49.904268 27584 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 14:46:49.904883 27356 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a7f46c254e64b2886924ddca7096198 in term 1.
I20260812 14:46:49.907519 27646 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7f46c254e64b2886924ddca7096198. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.907626 27646 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:49.907682 27646 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.907720 27646 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 14:46:49.907874 27653 mvcc.cc:205] Tried to move back new op lower bound from 7317692456503701504 to 7317692456358395904. Current Snapshot: MvccSnapshot[applied={T|T < 7317692456387309568}]
W20260812 14:46:49.908362 27645 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 14:46:49.908413 27645 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 14:46:49.908558 27626 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:49.908620 27626 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1 [sys.catalog]: This master's current role is: FOLLOWER
W20260812 14:46:49.921429 27262 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.919992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.15.227.254:40191)
0812 14:46:49.920033 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.920035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.920832 (+ 797us) server_negotiation.cc:407] Connection header received
0812 14:46:49.920926 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.920927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.920936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.920963 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.921373 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50784: BlockingRecv error: recv got EOF from 127.0.0.1:50784 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:46:49.922190 27556 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.920298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.15.227.252:34665)
0812 14:46:49.920333 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.920335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.921704 (+ 1369us) server_negotiation.cc:407] Connection header received
0812 14:46:49.921791 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.921792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.921801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.921826 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.922138 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46724: BlockingRecv error: recv got EOF from 127.0.0.1:46724 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260812 14:46:49.922933 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.920585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.15.227.253:38593)
0812 14:46:49.920623 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.920625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.922453 (+ 1828us) server_negotiation.cc:407] Connection header received
0812 14:46:49.922537 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.922538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.922547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.922572 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.922885 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60712: BlockingRecv error: recv got EOF from 127.0.0.1:60712 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:46:49.927379 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.926585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.15.227.253:38593)
0812 14:46:49.926637 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.926638 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.926886 (+ 248us) server_negotiation.cc:407] Connection header received
0812 14:46:49.926982 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.926983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.926998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.927024 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.927339 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60722: BlockingRecv error: recv got EOF from 127.0.0.1:60722 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 14:46:49.931069 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.930319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.15.227.253:38593)
0812 14:46:49.930349 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.930351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.930606 (+ 255us) server_negotiation.cc:407] Connection header received
0812 14:46:49.930705 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.930706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.930715 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.930741 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.931030 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60730: BlockingRecv error: recv got EOF from 127.0.0.1:60730 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:46:49.934834 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.934160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60742 (local address 127.15.227.253:38593)
0812 14:46:49.934190 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.934192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.934376 (+ 184us) server_negotiation.cc:407] Connection header received
0812 14:46:49.934460 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.934461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.934469 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.934495 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.934781 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60742: BlockingRecv error: recv got EOF from 127.0.0.1:60742 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:46:49.940577 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.939935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.15.227.253:38593)
0812 14:46:49.939964 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.939966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.940143 (+ 177us) server_negotiation.cc:407] Connection header received
0812 14:46:49.940229 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.940230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.940239 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.940264 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.940539 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60754: BlockingRecv error: recv got EOF from 127.0.0.1:60754 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260812 14:46:49.950408 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.949710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.15.227.253:38593)
0812 14:46:49.949741 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.949742 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.949930 (+ 188us) server_negotiation.cc:407] Connection header received
0812 14:46:49.950035 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.950036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.950046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.950072 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.950358 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60762: BlockingRecv error: recv got EOF from 127.0.0.1:60762 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 14:46:49.968274 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:49.967585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.15.227.253:38593)
0812 14:46:49.967615 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:49.967617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:49.967829 (+ 212us) server_negotiation.cc:407] Connection header received
0812 14:46:49.967914 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:49.967915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:49.967924 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:49.967949 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:49.968235 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60776: BlockingRecv error: recv got EOF from 127.0.0.1:60776 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260812 14:46:50.002336 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:50.001587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.15.227.253:38593)
0812 14:46:50.001638 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.001640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.001784 (+ 144us) server_negotiation.cc:407] Connection header received
0812 14:46:50.001825 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.001826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.001840 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.001907 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:50.002280 (+ 373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60778: BlockingRecv error: recv got EOF from 127.0.0.1:60778 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260812 14:46:50.067955 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:50.067265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.15.227.253:38593)
0812 14:46:50.067336 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.067338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.067432 (+ 94us) server_negotiation.cc:407] Connection header received
0812 14:46:50.067505 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.067506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.067518 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.067557 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:50.067892 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60788: BlockingRecv error: recv got EOF from 127.0.0.1:60788 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260812 14:46:50.197656 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:50.197024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.15.227.253:38593)
0812 14:46:50.197107 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.197109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.197171 (+ 62us) server_negotiation.cc:407] Connection header received
0812 14:46:50.197239 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.197240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.197251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.197302 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:50.197594 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60794: BlockingRecv error: recv got EOF from 127.0.0.1:60794 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 14:46:50.337582 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0a7f46c254e64b2886924ddca7096198" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a7f46c254e64b2886924ddca7096198" member_type: VOTER last_known_addr { host: "127.15.227.254" port: 40191 } } peers { permanent_uuid: "a8b044417baf49fa8767ce38ba0e68f1" member_type: VOTER last_known_addr { host: "127.15.227.253" port: 38593 } } peers { permanent_uuid: "57159baf2a2c4c64b0a6bfd201ade9d5" member_type: VOTER last_known_addr { host: "127.15.227.252" port: 34665 } } }
I20260812 14:46:50.337726 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7f46c254e64b2886924ddca7096198 [sys.catalog]: This master's current role is: LEADER
W20260812 14:46:50.455358 27408 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 14:46:50.454716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.15.227.253:38593)
0812 14:46:50.454778 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.454781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.454822 (+ 41us) server_negotiation.cc:407] Connection header received
0812 14:46:50.454904 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.454905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.454917 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.454962 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0812 14:46:50.455303 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60800: BlockingRecv error: recv got EOF from 127.0.0.1:60800 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260812 14:46:50.819516 27580 catalog_manager.cc:1259] Loaded cluster ID: 3a25b5015a9a421ea1691c89a2cc4460
I20260812 14:46:50.819581 27580 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: loading cluster ID for follower catalog manager: success
I20260812 14:46:50.820366 27580 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: acquiring CA information for follower catalog manager: success
I20260812 14:46:50.820657 27580 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 57159baf2a2c4c64b0a6bfd201ade9d5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:46:50.908857 27645 catalog_manager.cc:1259] Loaded cluster ID: 3a25b5015a9a421ea1691c89a2cc4460
I20260812 14:46:50.908937 27645 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: loading cluster ID for follower catalog manager: success
I20260812 14:46:50.909773 27645 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: acquiring CA information for follower catalog manager: success
I20260812 14:46:50.910135 27645 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a8b044417baf49fa8767ce38ba0e68f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 14:46:50.970674 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:50.968354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.15.227.253:38593)
0812 14:46:50.968471 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.968473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.968505 (+ 32us) server_negotiation.cc:407] Connection header received
0812 14:46:50.968571 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.968572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.968619 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.968691 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:50.968807 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.969299 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.969993 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.970219 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.970270 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:50.970399 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:50.970508 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:50.970563 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:50.970579 (+ 16us) server_negotiation.cc:299] Negotiation successful
0812 14:46:50.970595 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260812 14:46:50.974290 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:50.972311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.15.227.252:34665)
0812 14:46:50.972427 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.972429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.972437 (+ 8us) server_negotiation.cc:407] Connection header received
0812 14:46:50.972473 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.972474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.972510 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.972572 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:50.972641 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.973105 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.973717 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.973888 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.973928 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:50.974033 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:50.974131 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:50.974185 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:50.974215 (+ 30us) server_negotiation.cc:299] Negotiation successful
0812 14:46:50.974231 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260812 14:46:50.978514 27787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:50.975970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.15.227.252:34665)
0812 14:46:50.976015 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.976017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.976027 (+ 10us) server_negotiation.cc:407] Connection header received
0812 14:46:50.976086 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.976087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.976095 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.976151 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:50.976320 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.976835 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.977831 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.978131 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.978203 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:50.978335 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:50.978363 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:50.978405 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:50.978452 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 14:46:50.978468 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260812 14:46:50.978819 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:50.975766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.15.227.254:40191)
0812 14:46:50.975878 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.975879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.975909 (+ 30us) server_negotiation.cc:407] Connection header received
0812 14:46:50.975994 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.975995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.976048 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.976126 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:50.976260 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.976955 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.977893 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.978159 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.978218 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:50.978408 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:50.978506 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:50.978563 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:50.978706 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 14:46:50.978741 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":63,"threads_started":1}
I20260812 14:46:50.978834 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 14:46:50.975766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.15.227.253:38593)
0812 14:46:50.975806 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0812 14:46:50.975808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0812 14:46:50.976226 (+ 418us) server_negotiation.cc:407] Connection header received
0812 14:46:50.976315 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 14:46:50.976316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 14:46:50.976326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 14:46:50.976351 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 14:46:50.976515 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.977146 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.977870 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 14:46:50.978174 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 14:46:50.978207 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 14:46:50.978600 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 14:46:50.978632 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 14:46:50.978676 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0812 14:46:50.978761 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 14:46:50.978779 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260812 14:46:50.985013 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 27142
I20260812 14:46:51.001041 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 27270
I20260812 14:46:51.009425 16271 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskN3u55Q/build/release/bin/kudu with pid 27416
2026-08-12T14:46:51Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2929 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5888 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (270998 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST